Runway Documentation
  • About Runway
  • 🚀Getting started
    • Adding apps
    • Setting up your integrations
      • Branching strategies
      • Builds and branches
      • Pattern strings / tokens
    • Preparing your first release
    • Setting up your team
  • 🎛️Using Runway
    • Navigating Runway
    • Build Distro
      • Quickstart
      • Uploading builds for distribution
      • Build Distro buckets
      • Sharing builds
      • Installing builds
      • Signing and provisioning cheat sheet
    • App overview
    • Releases
      • Release overview
      • Feature flags
      • Kickoff
      • Feature readiness
      • Translations
      • Release candidate
      • Regression testing
      • Beta testing
      • Screenshots
      • Metadata
      • Approvals
      • App submission
      • App store review
      • Release
    • Release schedule
    • Rollout
    • Hotfixes
    • Rollbacks
    • Checklists
    • Build matching
    • App settings
      • General settings
      • Team settings
      • Release pilot rotation
      • Integrations settings
      • Profiles and devices
      • Signing keys
      • Health metrics settings
      • Automations settings
      • Notifications settings
      • Schedule settings
      • Beta testing settings
      • Metadata settings
      • Release defaults
      • Checklists settings
      • Custom strings settings
    • Organization overview
    • Organization settings
      • Team
      • SSO/SAML
    • User settings
    • Over-the-air (OTA) releases
  • ⚙️Integrations
    • Integrations overview
    • Version control
      • Azure Repos
      • Bitbucket
      • GitHub
      • GitLab
    • Project management
      • Asana
      • Azure Boards
      • GitHub Issues
      • Jira
      • Linear
      • Pivotal Tracker
      • Monday.com
      • Shortcut
    • CI/CD
      • App Center Build
      • Azure Pipelines
      • Bitbucket Pipelines
      • Bitrise
      • Buildkite
      • CircleCI
      • Codemagic
      • GitHub Actions
      • GitLab CI
      • Jenkins
      • TravisCI
      • Xcode Cloud
    • Regression testing
      • TestRail
      • Xray
    • Beta testing
      • Google Play testing tracks
      • TestFlight
      • App Center
      • Firebase App Distribution
    • App stores
      • App Store Connect
      • Google Play Console
      • Amazon Appstore
      • Huawei AppGallery
      • Samsung Galaxy Store
    • Notifications
      • Microsoft Teams
      • Slack
    • Stability monitoring
      • BugSnag
      • Datadog
      • Embrace
      • Firebase Crashlytics
      • New Relic
      • Sentry
      • Dynatrace
    • Observability & analytics
      • Amplitude
      • Datadog
      • Google Analytics
      • Mixpanel
      • New Relic
      • Custom observability & analytics
    • Feature flagging
      • Optimizely
      • LaunchDarkly
    • Scheduling
      • Opsgenie
      • PagerDuty
    • Translations
      • Crowdin
      • Lokalise
    • Calendar
      • Google Calendar
  • ⚡Automations
    • Automations overview
    • Types of automations
  • 🔔Notifications
    • Notifications overview
    • Types of notifications
  • 🪝Webhooks
    • Outgoing webhooks
  • 📡REST API
    • Overview
    • Reference Docs
    • Markdown Badge API
    • App configuration files
  • 🛠️Quickstart CI/CD
    • Quickstart CI/CD overview
    • Getting started
  • Support
    • Contact us
Powered by GitBook
On this page
  • Set up
  • Navigate to the Integration settings view
  • Connect TestRail

Was this helpful?

  1. Integrations
  2. Regression testing

TestRail

PreviousRegression testingNextXray

Last updated 1 year ago

Was this helpful?

Set up

Navigate to the Integration settings view

  1. Select an app in the top left corner from the Switcher

  2. Navigate to App Settings by clicking the gear icon (⚙️) at the top of the Timeline sidebar

  3. Click on Integrations in the sidebar

Connect TestRail

  • Find the TestRail integration module under the Regression testing section

  • Click the Connect button

  • Enter your TestRail instance’s base URL, including the https:// and excluding any trailing /

  • Enter a TestRail API Key

    • You can generate API Keys within TestRail by going to My Settings > API Keys > Add Key

The API key creation flow in TestRail can be confusing. Make sure you both generate and add the new API key, then click "Save Settings" (takes multiple separate clicks). If you refresh the main TestRail API keys page (My Settings > API Keys) and the key you created doesn't appear, you missed a click!

  • Click Save; you’ll be presented with a dialog in which to select your TestRail project and add any test run naming keywords

For the release linkage, Runway will parse your test run names for the presence of a version string (e.g. “2.3.0”)

You can optionally add test run naming keywords (case insensitive) to associate test runs with a particular app, e.g. "iOS" or "Android"

For more information on how Runway will leverage TestRail, visit the article.

⚙️
Regression testing step