Comment on page
Health metrics settings
Health metrics in Runway are configurable metrics you can use to monitor the health of a release as it rolls out. Runway supports a number of different metric types that are computed using data from your Stability Monitoring, Observability & Analytics and App store integrations.
Runway uses your team’s custom ‘health’ definition on the Rollout page to give you a glanceable, immediately understandable view of how your app is faring post-release.
Once health definitions have been created, they can be leveraged as rules for triggering automations that help manage your rollout, like halting unhealthy phased rollouts and accelerating healthy phased rollouts.
Runway will also send Slack notifications any time your defined health metrics fall outside of their thresholds during a release rollout.
Custom thresholds related to observability & analytics integrations can only be configured for the key events that you have selected in those integrations' settings.
Runway polls the connected stability monitoring integration to retrieve updated values for metrics every 10 minutes (approximately) during the course of a release’s rollout. Values are segmented by release version, platform, environment, and build number.
Supported metrics:
- Crash-free sessions
- Crash-free sessions delta from previous version value
- Crash-free sessions delta from previous version as a percentage
- Crash-free users
- Crash-free users delta from previous version value
- Crash-free users delta from previous version as a percentage
The percentage of sessions for the release that have not experienced a crash. Calculated as follows:
100 - (number of crashes that have occurred in a given release version / number of sessions for a release version * 100)
The percentage of users for the release that have not experienced a crash. Crash rate per user is calculated as follows:
Number of crashes that have occurred in a given release version divided by the number of unique users for a release version, multiplied by 100.
The delta between the current release's user stability or session stability value and the previous version's value.
The delta between the current release's user stability or session stability and the previous version's as a percent change.
Runway polls the connected observability & analytics integration to retrieve updated values for metrics every 10 minutes (approximately) during the course of a release’s rollout. Values are segmented by release version and platform.
Supported metrics:
- Events per DAU
- Events per DAU delta from previous version value
- Events per DAU delta from previous version as a percentage
- Events per session
- Events per session delta from previous version value
- Events per session delta from previous version as a percentage
- Average of property value
- Average of property value delta from previous version value
- Average of property value delta from previous version as a percentge
- Median of property value (supported by Datadog and Amplitude only)
- Median of property value delta from previous version value
- Median of property value delta from previous version as a percentage
- Percentiles of property value (p90, p95, p99) (supported by Datadog and Amplitude only)
- Percentiles of property value (p90, p95, p99) delta from previous version value
- Percentiles of property value (p90, p95, p99) delta from previous version as a percentage
The number of times an event is triggered per daily active user. Calculated by taking the total number of times an event has been triggered, and dividing that number by the total number of daily active users.
The number of times an event is triggered per session. Calculated by taking the total number of times an event has been triggered, and dividing that by the total number of sessions.
Events per DAU and events per session delta from previous version value
The delta between the current release's count of events per DAU / per session and the previous version's value.
The delta between the current release's count of events per DAU / per session and the previous version's value as a percentage.
For a given event, the average value of a specified property across all triggered events in the release version. For example:
Event name: "Add to cart"
Event property: "amount"
For all triggered "Add to cart" events in the release version, the average value of the "amount" property.
For a given event, the median value of a specific property across all triggered events in the release version.
Event name: "Add to cart"
Event property: "amount"
For all triggered "Add to cart" events in the release version, the median value of the "amount" property.
For a given event, the 90th, 95th, or 99th percentile values of a specific property across all triggered events in the release version.
Event name: "Add to cart"
Event property: "amount"
For all triggered "Add to cart" events in the release version, the 90th, 95th, and 99th percentiles of the "amount" property.
The delta between the current release's average, median, or percentile (p90, p95, p99) of property value and the previous version's values. For example:
Event name: "Add to cart"
Event property: "amount"
Average value: 95.00
Delta from previous version (value): +5.00
The delta between the current release's average, median, or percentile (p90, p95, p99) of property value and the previous version's as a percentage. For example:
Event name: "Add to cart"
Event property: "amount"
Average value: 95.00
Delta from previous version (percentage): +5.5%
Runway interacts with the relevant app store API to fetch the latest user rating average for a given release and app.
Supported metrics:
- User rating for version average
- User rating for version average delta from previous version
- User rating for version average delta from previous version as a percentage
The user rating for the given release version. This value comes directly from the Play Store or App Store integration, and is segmented by release version.
The delta between the current release's average user rating and the previous version's value. For example:
Average user rating (current version): 3.5
Delta from previous version (value): -0.5
The delta between the current release's avg user rating and the previous version's value as a percentage. For example:
Average user rating (current version): 3.5
Delta from previous version (percentage): -10%
Last modified 10d ago