Health metrics settings
Health metrics in Runway are configurable metrics you can use to monitor the health of a release as it rolls out. Runway supports a number of different metric types that are computed using data from your Stability Monitoring, Observability & Analytics and App store integrations.
Runway uses your team’s custom ‘health’ definition on the Rollout page to give you a glanceable, immediately understandable view of how your app is faring post-release.
Once health definitions have been created, they can be leveraged as rules for triggering automations that help manage your rollout, like halting unhealthy phased rollouts and accelerating healthy phased rollouts.
Runway will also send Slack notifications any time your defined health metrics fall outside of their thresholds during a release rollout.
Custom thresholds related to observability & analytics integrations can only be configured for the key events that you have selected in those integrations' settings.
Metric Types
Stability monitoring metric types
Runway polls the connected stability monitoring integration to retrieve updated values for metrics every 10 minutes (approximately) during the course of a release’s rollout. Values are segmented by release version, platform, environment, and build number.
Supported metrics:
Crash-free sessions
Crash-free sessions delta from previous version value
Crash-free sessions delta from previous version as a percentage
Crash-free users
Crash-free users delta from previous version value
Crash-free users delta from previous version as a percentage
Crash-free sessions
The percentage of sessions for the release that have not experienced a crash. Calculated as follows:
(Number of sessions on a given version - number of sessions on a given version that have experienced a crash) / Number of sessions on a given version
Crash-free users
The percentage of users for the release that have not experienced a crash. Crash rate per user is calculated as follows:
(Number of users on a given version - number of users on a given version that have experienced a crash) / Number of users on a given version
Crash-free sessions & users delta from previous version value
The delta between the current release's user stability or session stability value and the previous version's value.
Crash-free sessions & users delta from previous version as a percentage
The delta between the current release's user stability or session stability and the previous version's as a percent change.
Observability & Analytics metric types
Runway polls the connected observability & analytics integration to retrieve updated values for metrics every 10 minutes (approximately) during the course of a release’s rollout. Values are segmented by release version and platform.
Supported metrics:
Events per DAU
Events per DAU delta from previous version value
Events per DAU delta from previous version as a percentage
Events per session
Events per session delta from previous version value
Events per session delta from previous version as a percentage
Average of property value
Average of property value delta from previous version value
Average of property value delta from previous version as a percentge
Median of property value (supported by Datadog and Amplitude only)
Median of property value delta from previous version value
Median of property value delta from previous version as a percentage
Percentiles of property value (p90, p95, p99) (supported by Datadog and Amplitude only)
Percentiles of property value (p90, p95, p99) delta from previous version value
Percentiles of property value (p90, p95, p99) delta from previous version as a percentage
Events per DAU
The number of times an event is triggered per daily active user. Calculated by taking the total number of times an event has been triggered, and dividing that number by the total number of daily active users.
Events per session
The number of times an event is triggered per session. Calculated by taking the total number of times an event has been triggered, and dividing that by the total number of sessions.
Events per DAU and events per session delta from previous version value
The delta between the current release's count of events per DAU / per session and the previous version's value.
Events per DAU and events per session delta from previous version value as a percentage
The delta between the current release's count of events per DAU / per session and the previous version's value as a percentage.
Average of property value
For a given event, the average value of a specified property across all triggered events in the release version. For example:
Event name: "Add to cart"
Event property: "amount"
For all triggered "Add to cart" events in the release version, the average value of the "amount" property.
Median of property value
For a given event, the median value of a specific property across all triggered events in the release version.
Event name: "Add to cart"
Event property: "amount"
For all triggered "Add to cart" events in the release version, the median value of the "amount" property.
Percentiles of property value (p90, p95, p99)
For a given event, the 90th, 95th, or 99th percentile values of a specific property across all triggered events in the release version.
Event name: "Add to cart"
Event property: "amount"
For all triggered "Add to cart" events in the release version, the 90th, 95th, and 99th percentiles of the "amount" property.
Average, median, and percentile of property value delta from previous version
The delta between the current release's average, median, or percentile (p90, p95, p99) of property value and the previous version's values. For example:
Event name: "Add to cart"
Event property: "amount"
Average value: 95.00
Delta from previous version (value): +5.00
Average, median, and percentile of property value delta from previous version as percentage
The delta between the current release's average, median, or percentile (p90, p95, p99) of property value and the previous version's as a percentage. For example:
Event name: "Add to cart"
Event property: "amount"
Average value: 95.00
Delta from previous version (percentage): +5.5%
Ratings metric types
Runway interacts with the relevant app store API to fetch the latest user rating average for a given release and app.
Supported metrics:
User rating for version average
User rating for version average delta from previous version
User rating for version average delta from previous version as a percentage
User rating average
The user rating for the given release version. This value comes directly from the Play Store or App Store integration, and is segmented by release version.
User rating average delta from previous version
The delta between the current release's average user rating and the previous version's value. For example:
Average user rating (current version): 3.5
Delta from previous version (value): -0.5
User rating average delta from previous version as a percentage
The delta between the current release's avg user rating and the previous version's value as a percentage. For example:
Average user rating (current version): 3.5
Delta from previous version (percentage): -10%
Last updated