|
| 1 | +--- |
| 2 | +description: Learn how to get started with Pyroscope using a simple Ride share app. |
| 3 | +menuTitle: Ride share tutorial |
| 4 | +title: Ride share tutorial with Pyroscope |
| 5 | +weight: 250 |
| 6 | +killercoda: |
| 7 | + title: Ride share tutorial |
| 8 | + description: Learn how to get started with Pyroscope using a simple Ride share app. |
| 9 | + details: |
| 10 | + intro: |
| 11 | + foreground: docker-compose-update.sh |
| 12 | + backend: |
| 13 | + backend: |
| 14 | + imageid: ubuntu |
| 15 | +--- |
| 16 | + |
| 17 | +<!-- INTERACTIVE page intro.md START --> |
| 18 | + |
| 19 | +# Ride share tutorial with Pyroscope |
| 20 | + |
| 21 | +This tutorial demonstrates a basic use case of Pyroscope by profiling a "Ride Share" application. |
| 22 | +In this example, you learn: |
| 23 | + |
| 24 | +- How an application is instrumented with Pyroscope, including techniques for dynamically tagging functions. |
| 25 | +- How to view the resulting profile data in Grafana using the Profiles View. |
| 26 | +- How to integrate Pyroscope with Grafana to visualize the profile data. |
| 27 | + |
| 28 | +<!-- INTERACTIVE ignore START --> |
| 29 | + |
| 30 | +## Before you begin |
| 31 | + |
| 32 | +You need to have the following prerequisites to complete this tutorial: |
| 33 | +- Git |
| 34 | +- [Docker](https://docs.docker.com/compose/install/) |
| 35 | +- The Docker Compose plugin (included with Docker Desktop) |
| 36 | + |
| 37 | +{{< admonition type="tip" >}} |
| 38 | +Try this tutorial in an interactive learning environment: [Ride share tutorial with Pyroscope](https://killercoda.com/grafana-labs/course/pyroscope/ride-share-tutorial). |
| 39 | + |
| 40 | +It's a fully configured environment with all the dependencies installed. |
| 41 | + |
| 42 | +Provide feedback, report bugs, and raise issues in the [Grafana Killercoda repository](https://github.com/grafana/killercoda). |
| 43 | +{{< /admonition >}} |
| 44 | + |
| 45 | +<!-- INTERACTIVE ignore END --> |
| 46 | + |
| 47 | +## Background |
| 48 | + |
| 49 | +In this tutorial, you will profile a simple "Ride Share" application. The application is a Python Flask app that simulates a ride-sharing service. The app has three endpoints which are found in the `server.py` file: |
| 50 | + |
| 51 | +- `/bike` : calls the `order_bike(search_radius)` function to order a bike |
| 52 | +- `/car` : calls the `order_car(search_radius)` function to order a car |
| 53 | +- `/scooter` : calls the `order_scooter(search_radius)` function to order a scooter |
| 54 | + |
| 55 | +To simulate a highly available and distributed system, the app is deployed on three distinct servers in 3 different regions: |
| 56 | +- us-east |
| 57 | +- eu-north |
| 58 | +- ap-south |
| 59 | + |
| 60 | +This is simulated by running three instances of the server in Docker containers. Each server instance is tagged with the region it represents. |
| 61 | + |
| 62 | +{{< figure max-width="100%" src="/media/docs/pyroscope/ride-share-demo.gif" caption="Getting started sample application" alt="Getting started sample application" >}} |
| 63 | + |
| 64 | +In this scenario, a load generator will send mock-load to the three servers as well as their respective endpoints. This lets you see how the application performs per region and per vehicle type. |
| 65 | + |
| 66 | +{{<docs/ignore>}} |
| 67 | +{{< admonition type="tip" >}} |
| 68 | +A setup script runs in the background to install the necessary dependencies. This should take no longer than 30 seconds. Your instance will be ready to use once you `Setup complete. You may now begin the tutorial`. |
| 69 | +{{< /admonition >}} |
| 70 | +{{</docs/ignore>}} |
| 71 | + |
| 72 | +<!-- INTERACTIVE page intro.md END --> |
| 73 | + |
| 74 | +<!-- INTERACTIVE page step1.md START --> |
| 75 | + |
| 76 | +## Clone the repository |
| 77 | + |
| 78 | +1. Clone the repository to your local machine: |
| 79 | + |
| 80 | + ```bash |
| 81 | + git clone https://github.com/grafana/pyroscope.git && cd pyroscope |
| 82 | + ``` |
| 83 | + |
| 84 | +1. Navigate to the tutorial directory: |
| 85 | + |
| 86 | + ```bash |
| 87 | + cd examples/language-sdk-instrumentation/python/rideshare/flask |
| 88 | + ``` |
| 89 | + |
| 90 | +## Start the application |
| 91 | + |
| 92 | +Start the application using Docker Compose: |
| 93 | + |
| 94 | +```bash |
| 95 | +docker compose up -d |
| 96 | +``` |
| 97 | + |
| 98 | +This may take a few minutes to download the required images and build the demo application. Once ready, you will see the following output: |
| 99 | + |
| 100 | +```console |
| 101 | + ✔ Network flask_default Created |
| 102 | + ✔ Container flask-ap-south-1 Started |
| 103 | + ✔ Container flask-grafana-1 Started |
| 104 | + ✔ Container flask-pyroscope-1 Started |
| 105 | + ✔ Container flask-load-generator-1 Started |
| 106 | + ✔ Container flask-eu-north-1 Started |
| 107 | + ✔ Container flask-us-east-1 Started |
| 108 | +``` |
| 109 | + |
| 110 | +Optional: To verify the containers are running, run: |
| 111 | + |
| 112 | +```bash |
| 113 | +docker ps -a |
| 114 | +``` |
| 115 | +<!-- INTERACTIVE page step1.md END --> |
| 116 | + |
| 117 | +<!-- INTERACTIVE page step2.md START --> |
| 118 | + |
| 119 | +## Accessing Explore Profiles in Grafana |
| 120 | + |
| 121 | +Grafana includes the [Explore Profiles](https://grafana.com/docs/grafana/<GRAFANA_VERSION>/explore/simplified-exploration/profiles/) app that you can use to view profile data. To access Explore Profiles, open a browser and navigate to [http://localhost:3000/a/grafana-pyroscope-app/profiles-explorer](http://localhost:3000/a/grafana-pyroscope-app/profiles-explorer). |
| 122 | + |
| 123 | +### How tagging works |
| 124 | + |
| 125 | +In this example, the application is instrumented with Pyroscope using the Python SDK. |
| 126 | +The SDK allows you to tag functions with metadata that can be used to filter and group the profile data in the Explore Profiles. |
| 127 | +This example uses static and dynamic tagging. |
| 128 | + |
| 129 | +To start, let's take a look at a static tag use case. Within the `server.py` file, find the Pyroscope configuration: |
| 130 | +
|
| 131 | +```python |
| 132 | + pyroscope.configure( |
| 133 | + application_name = app_name, |
| 134 | + server_address = server_addr, |
| 135 | + basic_auth_username = basic_auth_username, # for grafana cloud |
| 136 | + basic_auth_password = basic_auth_password, # for grafana cloud |
| 137 | + tags = { |
| 138 | + "region": f'{os.getenv("REGION")}', |
| 139 | + } |
| 140 | + ) |
| 141 | +``` |
| 142 | +This tag is considered static is because the tag is set at the start of the application and doesn't change. |
| 143 | +In this case, it's useful for grouping profiles on a per region basis, which lets you see the performance of the application per region. |
| 144 | +
|
| 145 | +1. Open Grafana using the following url: [http://localhost:3000/a/grafana-pyroscope-app/profiles-explorer](http://localhost:3000/a/grafana-pyroscope-app/profiles-explorer). |
| 146 | +1. In the main menu, select **Explore** > **Profiles**. |
| 147 | +1. Select **Labels** in the **Exploration** path. |
| 148 | +1. Select the **region** tab in the **Group by labels** section. |
| 149 | +
|
| 150 | +You should now see a list of regions that the application is running in. You can see that `eu-north` is experiencing the most load. |
| 151 | +
|
| 152 | +{{< figure max-width="100%" src="/media/docs/pyroscope/ride-share-tag-region-2.png" caption="Region Tag" alt="Region Tag" >}} |
| 153 | +
|
| 154 | +Next, look at a dynamic tag use case. Within the `utils.py` file, find the following function: |
| 155 | +
|
| 156 | +```python |
| 157 | + def find_nearest_vehicle(n, vehicle): |
| 158 | + with pyroscope.tag_wrapper({ "vehicle": vehicle}): |
| 159 | + i = 0 |
| 160 | + start_time = time.time() |
| 161 | + while time.time() - start_time < n: |
| 162 | + i += 1 |
| 163 | + if vehicle == "car": |
| 164 | + check_driver_availability(n) |
| 165 | +``` |
| 166 | +
|
| 167 | +This example uses `tag_wrapper` to tag the function with the vehicle type. |
| 168 | +Notice that the tag is dynamic as it changes based on the vehicle type. |
| 169 | +This is useful for grouping profiles on a per vehicle basis. Allowing us to see the performance of the application per vehicle type being requested. |
| 170 | +
|
| 171 | +Use Explore Profiles to see how this tag is used: |
| 172 | +1. Open Explore Profiles using the following url: [http://localhost:3000/a/grafana-pyroscope-app/profiles-explorer](http://localhost:3000/a/grafana-pyroscope-app/profiles-explorer). |
| 173 | +1. Select on **Labels** in the **Exploration** path. |
| 174 | +1. In the **Group by labels** section, select the **vehicle** tab. |
| 175 | +
|
| 176 | +You should now see a list of vehicle types that the application is using. You can see that `car` is experiencing the most load. |
| 177 | +
|
| 178 | +<!-- INTERACTIVE page step2.md END --> |
| 179 | +
|
| 180 | +<!-- INTERACTIVE page step3.md START --> |
| 181 | +
|
| 182 | +## Identifying the performance bottleneck |
| 183 | +
|
| 184 | +The first step when analyzing a profile outputted from your application, is to take note of the largest node which is where your application is spending the most resources. |
| 185 | +To discover this, you can use the **Flame graph** view: |
| 186 | +
|
| 187 | +1. Open Explore Profiles using the following url: [http://localhost:3000/a/grafana-pyroscope-app/profiles-explorer](http://localhost:3000/a/grafana-pyroscope-app/profiles-explorer). |
| 188 | +1. Select **Flame graph** from the **Exploration** path. |
| 189 | +1. Verify that `flask-ride-sharing-app` is selected in the **Service** drop-down menu and `process_cpu/cpu` in the **Profile type** drop-down menu. |
| 190 | +
|
| 191 | +It should look something like this: |
| 192 | +
|
| 193 | +{{< figure max-width="100%" src="/media/docs/pyroscope/ride-share-bottle-neck-3.png" caption="Bottleneck" alt="Bottleneck" >}} |
| 194 | +
|
| 195 | +The flask `dispatch_request` function is the parent to three functions that correspond to the three endpoints of the application: |
| 196 | +- `order_bike` |
| 197 | +- `order_car` |
| 198 | +- `order_scooter` |
| 199 | +
|
| 200 | +By tagging both `region` and `vehicle` and looking at the [**Labels** view](https://grafana.com/docs/grafana/<GRAFANA_VERSION>/explore/simplified-exploration/profiles/choose-a-view/#labels), you can hypothesize: |
| 201 | +
|
| 202 | +- Something is wrong with the `/car` endpoint code where `car` vehicle tag is consuming **68% of CPU** |
| 203 | +- Something is wrong with one of our regions where `eu-north` region tag is consuming **54% of CPU** |
| 204 | +
|
| 205 | +From the flame graph, you can see that for the `eu-north` tag the biggest performance impact comes from the `find_nearest_vehicle()` function which consumes close to **68% of cpu**. |
| 206 | +To analyze this, go directly to the comparison page using the comparison dropdown. |
| 207 | +
|
| 208 | +### Comparing two time periods |
| 209 | +
|
| 210 | +The **Diff flame graph** view lets you compare two time periods side by side. |
| 211 | +This is useful for identifying changes in performance over time. |
| 212 | +This example compares the performance of the `eu-north` region within a given time period against the other regions. |
| 213 | +
|
| 214 | +1. Open Explore Profiles in Grafana using the following url: [http://localhost:3000/a/grafana-pyroscope-app/profiles-explorer](http://localhost:3000/a/grafana-pyroscope-app/profiles-explorer). |
| 215 | +1. Select **Diff flame graph** in the **Exploration** path. |
| 216 | +1. In **Baseline**, filter by `region` and select `!= eu-north`. |
| 217 | +1. In **Comparison**, filter by `region` and select `== eu-north`. |
| 218 | +1. In **Baseline**, select the time period you want to compare against. |
| 219 | + |
| 220 | +Scroll down to compare the two time periods side by side. |
| 221 | +Note that the `eu-north` region (right side) shows an excessive amount of time spent in the `find_nearest_vehicle` function. |
| 222 | +This looks to be caused by a mutex lock that is causing the function to block. |
| 223 | +
|
| 224 | +{{< figure max-width="100%" src="/media/docs/pyroscope/ride-share-time-comparison-2.png" caption="Time Comparison" alt="Time Comparison" >}} |
| 225 | +
|
| 226 | +<!-- INTERACTIVE page step3.md END --> |
| 227 | +
|
| 228 | +<!-- INTERACTIVE page step4.md START --> |
| 229 | +
|
| 230 | +## How was Pyroscope integrated with Grafana in this tutorial? |
| 231 | +
|
| 232 | +The `docker-compose.yml` file includes a Grafana container that's pre-configured with the Pyroscope plugin: |
| 233 | + |
| 234 | +```yaml |
| 235 | + grafana: |
| 236 | + image: grafana/grafana:latest |
| 237 | + environment: |
| 238 | + - GF_INSTALL_PLUGINS=grafana-pyroscope-app |
| 239 | + - GF_AUTH_ANONYMOUS_ENABLED=true |
| 240 | + - GF_AUTH_ANONYMOUS_ORG_ROLE=Admin |
| 241 | + - GF_AUTH_DISABLE_LOGIN_FORM=true |
| 242 | + volumes: |
| 243 | + - ./grafana-provisioning:/etc/grafana/provisioning |
| 244 | + ports: |
| 245 | + - 3000:3000 |
| 246 | +``` |
| 247 | + |
| 248 | +Grafana is also pre-configured with the Pyroscope data source. |
| 249 | + |
| 250 | +### Challenge |
| 251 | + |
| 252 | +As a challenge, see if you can generate a similar comparison with the `vehicle` tag. |
| 253 | + |
| 254 | +<!-- INTERACTIVE page step4.md END --> |
| 255 | + |
| 256 | +<!-- INTERACTIVE page finish.md START --> |
| 257 | + |
| 258 | +## Summary |
| 259 | + |
| 260 | +In this tutorial, you learned how to profile a simple "Ride Share" application using Pyroscope. |
| 261 | +You have learned some of the core instrumentation concepts such as tagging and how to use Explore Profiles identify performance bottlenecks. |
| 262 | + |
| 263 | +### Next steps |
| 264 | + |
| 265 | +- Learn more about the Pyroscope SDKs and how to [instrument your application with Pyroscope](https://grafana.com/docs/pyroscope/<PYROSCOPE_VERSION>/configure-client/). |
| 266 | +- Deploy Pyroscope in a production environment using the [Pyroscope Helm chart](https://grafana.com/docs/pyroscope/<PYROSCOPE_VERSION>/deploy-kubernetes/). |
| 267 | +- Continue exploring your profile data using [Explore Profiles](https://grafana.com/docs/grafana/<GRAFANA_VERSION>/explore/simplified-exploration/profiles/investigate/) |
| 268 | +<!-- INTERACTIVE page finish.md END --> |
| 269 | + |
| 270 | + |
| 271 | + |
| 272 | + |
| 273 | + |
| 274 | + |
| 275 | + |
| 276 | + |
| 277 | + |
0 commit comments