Runway Documentation
  • About Runway
  • 🚀Getting started
    • Adding apps
    • Setting up your integrations
      • Branching strategies
      • Builds and branches
      • Pattern strings / tokens
    • Preparing your first release
    • Setting up your team
  • 🎛️Using Runway
    • Navigating Runway
    • Build Distro
      • Quickstart
      • Uploading builds for distribution
      • Build Distro buckets
      • Sharing builds
      • Installing builds
      • Signing and provisioning cheat sheet
    • App overview
    • Releases
      • Release overview
      • Feature flags
      • Kickoff
      • Feature readiness
      • Translations
      • Release candidate
      • Regression testing
      • Beta testing
      • Screenshots
      • Metadata
      • Approvals
      • App submission
      • App store review
      • Release
    • Release schedule
    • Rollout
    • Hotfixes
    • Rollbacks
    • Checklists
    • Build matching
    • App settings
      • General settings
      • Team settings
      • Release pilot rotation
      • Integrations settings
      • Profiles and devices
      • Signing keys
      • Health metrics settings
      • Automations settings
      • Notifications settings
      • Schedule settings
      • Beta testing settings
      • Metadata settings
      • Release defaults
      • Checklists settings
      • Custom strings settings
    • Organization overview
    • Organization settings
      • Team
      • SSO/SAML
    • User settings
    • Over-the-air (OTA) releases
  • ⚙️Integrations
    • Integrations overview
    • Version control
      • Azure Repos
      • Bitbucket
      • GitHub
      • GitLab
    • Project management
      • Asana
      • Azure Boards
      • GitHub Issues
      • Jira
      • Linear
      • Pivotal Tracker
      • Monday.com
      • Shortcut
    • CI/CD
      • App Center Build
      • Azure Pipelines
      • Bitbucket Pipelines
      • Bitrise
      • Buildkite
      • CircleCI
      • Codemagic
      • GitHub Actions
      • GitLab CI
      • Jenkins
      • TravisCI
      • Xcode Cloud
    • Regression testing
      • TestRail
      • Xray
    • Beta testing
      • Google Play testing tracks
      • TestFlight
      • App Center
      • Firebase App Distribution
    • App stores
      • App Store Connect
      • Google Play Console
      • Amazon Appstore
      • Huawei AppGallery
      • Samsung Galaxy Store
    • Notifications
      • Microsoft Teams
      • Slack
    • Stability monitoring
      • BugSnag
      • Datadog
      • Embrace
      • Firebase Crashlytics
      • New Relic
      • Sentry
      • Dynatrace
    • Observability & analytics
      • Amplitude
      • Datadog
      • Google Analytics
      • Mixpanel
      • New Relic
      • Custom observability & analytics
    • Feature flagging
      • Optimizely
      • LaunchDarkly
    • Scheduling
      • Opsgenie
      • PagerDuty
    • Translations
      • Crowdin
      • Lokalise
    • Calendar
      • Google Calendar
  • ⚡Automations
    • Automations overview
    • Types of automations
  • 🔔Notifications
    • Notifications overview
    • Types of notifications
  • 🪝Webhooks
    • Outgoing webhooks
  • 📡REST API
    • Overview
    • Reference Docs
    • Markdown Badge API
    • App configuration files
  • 🛠️Quickstart CI/CD
    • Quickstart CI/CD overview
    • Getting started
  • Support
    • Contact us
Powered by GitBook
On this page
  • Beta soak status
  • Active build with a beta testing integration
  • Beta testing automations
  • App submission checklist

Was this helpful?

  1. Using Runway
  2. Releases

Beta testing

PreviousRegression testingNextScreenshots

Last updated 3 years ago

Was this helpful?

The Beta testing step shows the status of beta testing for the active Release Candidate. If a beta testing integration (TestFlight or Google Play testing track) is active, additional information such as beta testing history, and test details (testers, testing groups, beta testing release notes) will be shown.

Beta soak status

The active build module shows the current status of your beta soak (i.e. a minimum period of time allocated to verify that the stability and functionality of your release candidate meets expectations).

You can change your preferred beta testing style from either , or from the Change beta style link on the Active build module found on the beta testing step.

  • Non-blocking — No beta soak period. The Beta testing step will automatically be shown as complete (green).

  • Open-ended soak — An open-ended beta testing soak period. The Beta testing step will be marked as complete once the soak period is manually ended.

  • Time-limited soak — Set a beta soak duration, in hours. The Beta testing step will be marked as complete once the soak duration has finished. (You can still manually end the beta soak period early.)

Changing beta style from an individual release will apply that setting to all future releases.

The active build’s status will show as In testing while a beta soak period is ongoing, and Testing complete when the soak period has ended or was set as non-blocking.

Active build with a beta testing integration

If your team has connected a beta testing integration (TestFlight or Google Play testing tracks), Runway will additionally pull in several more details about your beta testing parameters.

  • (iOS) Review status — If you have external testers participating in beta testing via TestFlight, Apple requires an initial review. Runway will show you the status of your beta review: Needs review, In review, or In testing (the latter means that your build has been approved for external testers). You can manually submit each build for beta review via the button in the active build module, or enable the automation.

  • (iOS) dSYMs — Links are surfaced here to download dSYMs from App Store Connect. If the automation is enabled, Runway will also show the status of that upload here.

  • What to test / Release notes — Let your testers know what you would like them to test in this build. This information will be available to testers in all groups who have access to this build.

  • Testers — An additional tab will be active that displays all testers added for beta testing. Use the dropdown menu to view members of specific groups.

  • (iOS) Groups — Here, you can enable access to this beta build for additional predefined testing groups. Currently, you’ll need to visit App Store Connect directly to create new Groups, edit existing Groups, or remove Groups that currently have access. All eligible builds are automatically available to users in the App Store Connect Users group (these are considered ‘internal testers’ by Apple).

  • (iOS) Individual testers — View a list of any individual testers (i.e. testers that are not part of a Group) that have been given access to this beta test. Currently, you’ll need to visit App Store Connect directly to edit this list.

History

At the bottom of the view, Runway shows a count and a list of all builds (including final testing status) that have previously been tested for this version release. Expand each row to see further information (dSYMs, release notes, testers).

Beta testing automations

You can configure the following App submission automations by visiting the Automations tab on this step:

App submission checklist

You can add checklist items to this step by visiting the Checklist tab.

cover any unique parts of your team's release process and live across all your releases. Steps with checklist items won't be marked as complete (green) in Runway until all checklist items have been completed (in addition to the normal criteria that would mark a step as completed).

🎛️
Checklist items
App settings
Submit new builds for beta review
Download dSYMs from App Store Connect and upload to stability monitoring
(iOS) Assign beta testing builds to default testing groups
(iOS) Submit new builds for beta review
(iOS) Apply default Export Compliance Information to new builds in App Store Connect
(iOS) Download dSYMs from App Store Connect and upload to stability monitoring
(Android) Promote new builds to testing track