Bitrise regression testing

Start from a test plan. Bitrise uploads the build or calls the API, then runs that plan. For a change review of a pull request build, use Bitrise pull request testing.

Set it up

Create a test plan

Pick the tests, then copy the plan short ID (pln_…).

Mobile build

Upload the APK or simulator app and run the plan against it.

Web app

Start the plan with one POST /v1/run request.

Blocking mode

Poll the run and fail the step when the result is not passed.

What works on Bitrise

Capability	Bitrise
Review on a preview URL, before merge	No. Bitrise does not post a pull request review.
Review after merge, with no preview URL	No. See Bitrise pull request testing if you call the change review API yourself.
Run a regression test plan from CI	Yes. Mobile: upload the build and start the plan against it. Web: one HTTP request with the test plan short ID.
Run that plan against a mobile build	Yes. This is the primary Bitrise path.
Block the workflow until the plan finishes	Yes. Poll the run and fail the step when the result is not passed.
Schedule	On the test plan, with Manage Schedules. Bitrise does not have to be involved.

What this does not do

The scripts on the Bitrise page do not post a GitHub or GitLab review.
Bitrise does not replace a test plan schedule. Manage Schedules runs the plan without a Bitrise build.
Envoyer is unrelated. It is a PHP deploy hook, not a mobile CI. See Envoyer.

Test your mobile builds

Add a script step after your build step (e.g. android-build or xcode-build-for-simulator). It uploads the build to QA.tech and starts a test run against it, in four parts:

Get a presigned upload URL
Upload the build file directly to storage
Create the build record
Start a test run pinned to that build

Android (APK)

Bitrise’s android-build step exposes the built APK as $BITRISE_APK_PATH:

- script@1:
    title: Run QA.tech tests on this build
    inputs:
      - content: |
          #!/usr/bin/env bash
          set -euo pipefail

          APP_ID="app_gXeBl2"               # Your QA.tech application short ID
          TEST_PLAN_ID="pln_abc123"         # Your test plan short ID
          BUILD_FILE="$BITRISE_APK_PATH"    # Set by the android-build step
          FILE_NAME=$(basename "$BUILD_FILE")

          # 1. Get a presigned upload URL
          UPLOAD_RESPONSE=$(curl -sSf -X POST "https://api.qa.tech/v1/applications/$APP_ID/builds/upload-url" \
            -H "Authorization: Bearer $QATECH_API_TOKEN" \
            -H "Content-Type: application/json" \
            -d "{\"fileName\": \"$FILE_NAME\"}")
          UPLOAD_URL=$(echo "$UPLOAD_RESPONSE" | jq -r '.uploadUrl')
          BUILD_TOKEN=$(echo "$UPLOAD_RESPONSE" | jq -r '.buildToken')

          # 2. Upload the file directly to storage
          curl -sSf -X PUT "$UPLOAD_URL" \
            --upload-file "$BUILD_FILE" \
            -H "Content-Type: application/octet-stream"

          # 3. Create the build record
          BUILD_RESPONSE=$(curl -sSf -X POST "https://api.qa.tech/v1/applications/$APP_ID/builds" \
            -H "Authorization: Bearer $QATECH_API_TOKEN" \
            -H "Content-Type: application/json" \
            -d "{\"platform\": \"android\", \"buildToken\": \"$BUILD_TOKEN\"}")
          BUILD_SHORT_ID=$(echo "$BUILD_RESPONSE" | jq -r '.applicationBuildShortId')
          echo "Build created: $BUILD_SHORT_ID"

          # 4. Start a test run against this build
          RUN_RESPONSE=$(curl -sSf -X POST "https://api.qa.tech/v1/run" \
            -H "Authorization: Bearer $QATECH_API_TOKEN" \
            -H "Content-Type: application/json" \
            -d "{
              \"testPlanShortId\": \"$TEST_PLAN_ID\",
              \"applications\": [{
                \"applicationShortId\": \"$APP_ID\",
                \"environment\": {
                  \"applicationBuildShortId\": \"$BUILD_SHORT_ID\"
                }
              }]
            }")
          echo "Test run started: $(echo "$RUN_RESPONSE" | jq -r '.run.url')"

Replace app_gXeBl2 and pln_abc123 with your values.

iOS (Simulator build)

QA.tech runs iOS tests on simulators, so the upload must be a simulator build (.app compressed as .zip or .tar.gz) - device and App Store .ipa builds cannot run on simulators. See Mobile App Testing for how to prepare a simulator build. On Bitrise, use the xcode-build-for-simulator step instead of xcode-archive. It exposes the built .app directory as $BITRISE_APP_DIR_PATH. Zip it before the upload in the script above:

cd "$(dirname "$BITRISE_APP_DIR_PATH")"
zip -r app-simulator.zip "$(basename "$BITRISE_APP_DIR_PATH")"
BUILD_FILE="$PWD/app-simulator.zip"

and use "platform": "ios" when creating the build record.

Supported file types are .apk and .aab for Android, and .zip or .tar.gz containing your .app simulator build for iOS. Maximum file size is 4GB. See the Application Builds API for full request and response details.

Web applications

If you use Bitrise for a web app, trigger a test plan with a single request:

- script@1:
    title: Trigger QA.tech tests
    inputs:
      - content: |
          #!/usr/bin/env bash
          set -euo pipefail
          curl -sSf -X POST "https://api.qa.tech/v1/run" \
            -H "Authorization: Bearer $QATECH_API_TOKEN" \
            -H "Content-Type: application/json" \
            -d '{"testPlanShortId": "pln_abc123"}'

See the Start Run API for all available options, including environment URL overrides for staging or preview deployments.

Blocking mode

To fail the Bitrise build when tests fail (for example as a release gate), poll the run status after starting it:

# Start run and capture shortId
RUN_RESPONSE=$(curl -sSf -X POST "https://api.qa.tech/v1/run" \
  -H "Authorization: Bearer $QATECH_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d "{
    \"testPlanShortId\": \"$TEST_PLAN_ID\",
    \"applications\": [{
      \"applicationShortId\": \"$APP_ID\",
      \"environment\": { \"applicationBuildShortId\": \"$BUILD_SHORT_ID\" }
    }]
  }")
SHORT_ID=$(echo "$RUN_RESPONSE" | jq -r '.run.shortId')

# Poll until completion
while true; do
  STATUS_RESPONSE=$(curl -sSf "https://api.qa.tech/v1/run/$SHORT_ID" \
    -H "Authorization: Bearer $QATECH_API_TOKEN")
  STATUS=$(echo "$STATUS_RESPONSE" | jq -r '.status')
  if [[ "$STATUS" == "COMPLETED" || "$STATUS" == "ERROR" || "$STATUS" == "CANCELLED" ]]; then
    RESULT=$(echo "$STATUS_RESPONSE" | jq -r '.result')
    [[ "$RESULT" == "PASSED" ]] && exit 0 || exit 1
  fi
  sleep 30
done

See the Run Status API for polling details and error handling. If the polling step might exceed your step timeout, raise the step’s timeout in the Bitrise Workflow Editor.

Regression Testing

Agents

Choose your CI

Bitrise regression testing

Set it up

Create a test plan

Mobile build

Web app

Blocking mode

What works on Bitrise

What this does not do

Test your mobile builds

Android (APK)

iOS (Simulator build)

Web applications

Blocking mode

​Set it up

Create a test plan

Mobile build

Web app

Blocking mode

​What works on Bitrise

​What this does not do

​Test your mobile builds

​Android (APK)

​iOS (Simulator build)

​Web applications

​Blocking mode

Set it up

What works on Bitrise

What this does not do

Test your mobile builds

Android (APK)

iOS (Simulator build)

Web applications

Blocking mode