GitLab regression testing

Start from a test plan. GitLab CI calls the API to run that plan. It does not choose the tests. For a review of the merge request diff, use GitLab pull request testing.

Set it up

Create a test plan

Pick the tests, then copy the plan short ID (pln_…).

GitLab CI

Store the token as a CI/CD variable and call POST /v1/run.

Blocking mode

Poll the run and fail the job when the result is not passed.

Mobile build

Upload the build and run the plan against it.

What works on GitLab

Capability	GitLab
Review on a preview URL, before merge	Not this guide. A test plan run does not post a merge request review.
Review after merge, with no preview URL	Not this guide. See GitLab pull request testing.
Run a regression test plan from CI	Yes. `POST /v1/run` with the test plan short ID. You can override the URL for a preview or staging deploy.
Run that plan against a mobile build	Yes. Upload the build and pass `applicationBuildShortId`. See Mobile regression testing.
Block the pipeline until the plan finishes	Yes. Poll the run and fail the job when the result is not passed.
Schedule	On the test plan, with Manage Schedules. GitLab can also cron a pipeline that calls the same API.

What this does not do

This path does not post a merge request review. Use GitLab pull request testing for that.
GitLab cannot promote review tests into the suite. That agent exists on the GitHub App only.
Envoyer is not required. Use it only when the deploy itself is what should start the plan. See Envoyer.

API implementation patterns

Basic setup

trigger_qatech:
  stage: test
  variables:
    QATECH_TEST_PLAN_SHORT_ID: 'pln_abc123'
  script:
    - >-
      jq -n
      --arg testPlanShortId "$QATECH_TEST_PLAN_SHORT_ID"
      --arg actor "$GITLAB_USER_LOGIN"
      --arg branch "$CI_COMMIT_REF_NAME"
      --arg commitHash "$CI_COMMIT_SHA"
      --arg repository "$CI_PROJECT_PATH"
      --arg repositoryUrl "$CI_PROJECT_URL"
      '{ trigger: "GITLAB", testPlanShortId: $testPlanShortId, actor: $actor, branch: $branch, commitHash: $commitHash, repository: $repository, repositoryUrl: $repositoryUrl }'
      > qatech-request.json
    - >-
      curl --fail-with-body
      --request POST
      --url "https://api.qa.tech/v1/run"
      --header "Authorization: Bearer $QA_TECH_API_TOKEN"
      --header "Content-Type: application/json"
      --data @qatech-request.json

Replace pln_abc123 with your test plan short ID (from your test plan page). The runner image must provide curl and jq. The GITLAB trigger and repository metadata attribute the run to GitLab in QA.tech. The results page links the commit to $CI_PROJECT_URL/-/commit/$CI_COMMIT_SHA, including for self-hosted GitLab projects. Requests without this metadata remain generic API runs.

Run test plans on merge requests

test_mr:
  stage: test
  only:
    - merge_requests
  variables:
    QATECH_TEST_PLAN_SHORT_ID: 'pln-smoke-tests_abc123'
  script:
    - >-
      jq -n
      --arg testPlanShortId "$QATECH_TEST_PLAN_SHORT_ID"
      --arg actor "$GITLAB_USER_LOGIN"
      --arg branch "$CI_COMMIT_REF_NAME"
      --arg commitHash "$CI_COMMIT_SHA"
      --arg repository "$CI_PROJECT_PATH"
      --arg repositoryUrl "$CI_PROJECT_URL"
      '{ trigger: "GITLAB", testPlanShortId: $testPlanShortId, actor: $actor, branch: $branch, commitHash: $commitHash, repository: $repository, repositoryUrl: $repositoryUrl }'
      > qatech-request.json
    - >-
      curl --fail-with-body
      --request POST
      --url "https://api.qa.tech/v1/run"
      --header "Authorization: Bearer $QA_TECH_API_TOKEN"
      --header "Content-Type: application/json"
      --data @qatech-request.json

Test preview deployments via API

Pass dynamic URLs between jobs using dotenv artifacts:

stages:
  - deploy
  - test

deploy_preview:
  stage: deploy
  script:
    - echo "PREVIEW_URL=https://preview-${CI_MERGE_REQUEST_IID}.yourdomain.com" >> deploy.env
  artifacts:
    reports:
      dotenv: deploy.env

test_preview:
  stage: test
  dependencies:
    - deploy_preview
  before_script:
    - |
      echo '{"testPlanShortId":"pln-regression-suite_abc123","applications":[{"applicationShortId":"app-frontend_abc123","environment":{"url":"'$PREVIEW_URL'","name":"MR-'$CI_MERGE_REQUEST_IID'"}}]}' > /tmp/request.json
  script: >
    curl --request POST
    --url "https://api.qa.tech/v1/run"
    --header "Authorization: Bearer $QA_TECH_API_TOKEN"
    --header "Content-Type: application/json"
    --data @/tmp/request.json

The environment object also accepts optional customHeaders to persist auth or protection-bypass headers on that environment. Omit the field to leave stored headers unchanged; pass [] to clear them. Add devicePresetShortId on each application object in the same request. See Environment custom headers and Start Run API.

Scheduled testing

nightly_tests:
  stage: test
  only:
    - schedules
  variables:
    QATECH_REQUEST_BODY: '{"testPlanShortId": "pln-full-regression_abc123"}'
  script: >
    curl --request POST
    --url "https://api.qa.tech/v1/run"
    --header "Authorization: Bearer $QA_TECH_API_TOKEN"
    --header "Content-Type: application/json"
    --data "${QATECH_REQUEST_BODY}"

Set up schedules at CI/CD → Schedules → New schedule.

GitLab Schedules vs QA.tech Schedules

Use GitLab schedules when:

Tests should run as part of your CI/CD pipeline
You need GitLab context (branch, commit SHA)
You want to gate deployments on scheduled test results

Use QA.tech schedules when:

Tests should run independently of CI/CD infrastructure
You prefer managing schedules in QA.tech UI
You want to avoid consuming GitLab runner minutes

See Running Tests for QA.tech scheduling.

Blocking mode

Wait for test completion before proceeding with deployments:

trigger_qatech:
  stage: test
  script:
    # Start run and capture shortId
    - |
      RESPONSE=$(curl -s -X POST \
        "https://api.qa.tech/v1/run" \
        -H "Authorization: Bearer $QA_TECH_API_TOKEN" \
        -H "Content-Type: application/json" \
        -d "{\"testPlanShortId\": \"pln_abc123\"}")
      SHORT_ID=$(echo "$RESPONSE" | jq -r '.run.shortId')

    # Poll until completion (see Run Status API for details)
    - |
      while true; do
        RESPONSE=$(curl -s \
          "https://api.qa.tech/v1/run/$SHORT_ID" \
          -H "Authorization: Bearer $QA_TECH_API_TOKEN")
        STATUS=$(echo "$RESPONSE" | jq -r '.status')
        if [[ "$STATUS" == "COMPLETED" || "$STATUS" == "ERROR" || "$STATUS" == "CANCELLED" ]]; then
          RESULT=$(echo "$RESPONSE" | jq -r '.result')
          [[ "$RESULT" == "PASSED" ]] && exit 0 || exit 1
        fi
        sleep 30
      done

See Run Status API for polling logic details and error handling.

Custom Slack notifications

Override the notification channel for one run. See Per-run overrides.

Regression Testing

Agents

Choose your CI

GitLab regression testing

Set it up

Create a test plan

GitLab CI

Blocking mode

Mobile build

What works on GitLab

What this does not do

API implementation patterns

Basic setup

Run test plans on merge requests

Test preview deployments via API

Scheduled testing

Blocking mode

Custom Slack notifications

​Set it up

Create a test plan

GitLab CI

Blocking mode

Mobile build

​What works on GitLab

​What this does not do

​API implementation patterns

​Basic setup

​Run test plans on merge requests

​Test preview deployments via API

​Scheduled testing

​Blocking mode

​Custom Slack notifications

Set it up

What works on GitLab

What this does not do

API implementation patterns

Basic setup

Run test plans on merge requests

Test preview deployments via API

Scheduled testing

Blocking mode

Custom Slack notifications