> ## Documentation Index
> Fetch the complete documentation index at: https://docs.qa.tech/llms.txt
> Use this file to discover all available pages before exploring further.

# GitHub regression testing

> Run a test plan from GitHub Actions after a deploy, on a schedule, or as a blocking check.

Start from a [test plan](/core-concepts/test-plans). GitHub Actions runs that plan. It does not choose the tests. For a review of the pull request diff, use [GitHub pull request testing](/pr-testing/github).

## Set it up

<CardGroup cols={2}>
  <Card title="Create a test plan" icon="clipboard-list" href="/core-concepts/test-plans">
    Pick the tests, then copy the plan short ID (`pln_…`).
  </Card>

  <Card title="Test Run Action" icon="github" href="/configuration/github-actions#test-run-action">
    Run the plan from a workflow, with blocking and preview URL overrides.
  </Card>

  <Card title="Mobile build" icon="mobile" href="/regression-testing/agents/mobile#run-a-test-plan-on-the-pr">
    Upload the build and run the plan against it.
  </Card>

  <Card title="Schedule" icon="clock" href="/best-practices/running-tests#trigger-a-run-on-a-schedule">
    Run the plan on a cron schedule from **Manage Schedules**.
  </Card>
</CardGroup>

The [post-merge agent](/configuration/github-app#post-merge-agent) is separate. After a reviewed pull request merges, it can promote tests from that review into the suite. It does not start the test plan.

## What works on GitHub

| Capability                                 | GitHub                                                                                                                        |
| :----------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------- |
| Review on a preview URL, before merge      | Not this guide. The Test Run Action does not post a pull request review.                                                      |
| Review after merge, with no preview URL    | Not this guide. See [GitHub pull request testing](/pr-testing/github).                                                        |
| Run a regression test plan from CI         | Yes. The Test Run Action runs the plan you name, including against a preview URL you pass in.                                 |
| Run that plan against a mobile build       | Yes. Upload the build and pass `applicationBuildShortId`. See [Mobile regression testing](/regression-testing/agents/mobile). |
| Block the workflow until the plan finishes | Yes. Set blocking on the Test Run Action.                                                                                     |
| Schedule                                   | On the test plan, with **Manage Schedules**. GitHub is not required for that.                                                 |

## What this does not do

* The Test Run Action does not post a pull request review and does not pick tests from the diff.
* Envoyer is a different trigger for the same kind of plan. It is not required when you have GitHub Actions. See [Envoyer](/regression-testing/envoyer).
* A schedule configured only in GitHub Actions is optional. The product schedule is **Manage Schedules** on the test plan.

## Post-merge agent

When a pull request is **merged**, QA.tech can run an autonomous **post-merge agent** to keep your test suite healthy — before the internal review retrospective runs. Configure it under **Settings → Integrations → GitHub App** with the **Run post-merge agent** toggle, which reveals a **Post-merge prompt** you can edit. These settings live on the GitHub App integration (alongside Auto-run on PRs and other merge-review options).

**New GitHub integrations** default the agent on with the default prompt prefilled. **Existing integrations** keep the agent off until you enable it (the default prompt is still prefilled in the form).

You can also pass per-PR options when starting a change review via the [Start change review chat API](/api-reference/chat/start-change-review-chat) or the [Change Review Action](/configuration/github-actions#change-review-action). See [API and CI options](#post-merge-via-api-or-ci) below.

The key idea: **the prompt is the agent's entire goal.** The agent only changes tests when the prompt asks it to. If you point the prompt at something else (for example, "open a Linear issue summarizing the merge"), that is all it does — it will not touch your tests. The default prompt focuses on conservative regression maintenance.

The agent runs in the same conversation as the PR review, so it already has the review history — the tests that ran, their results, and the diff — as context. It reuses your existing test tools; there is nothing new to configure beyond the prompt.

### What it can do

Depending on your prompt, the agent can:

* **Promote review tests** into the regression suite. Only tests this PR's own reviews created — surfaced to the agent as "main candidates," still drafts and not last seen failing — can be promoted. Promoting activates a test (and its dependencies) and labels it `regression` + `auto-added` automatically, so you can filter for them in the test list. Promoting nothing is a normal outcome.
* **Update** tests whose behavior the merge changed so they match the merged state.
* **Convert to draft** tests that the merge made stale or irrelevant — cautiously, and only with strong, diff-grounded evidence. If another active test depends on one being drafted, the agent must confirm and draft the whole chain together; it never archives tests automatically.
* **Add tests to a test plan** (for example Smoke or Regression) when your prompt asks for it and defines the criteria.
* **Use connected integrations** — for example open a tracker issue, post or read a Slack update (when **Enable Slack Tools in Chat** is on), or (when you have connected MCP servers) run one of their tools — when your prompt asks for something other than test maintenance.

Before promoting or creating anything, the agent checks for an equivalent existing test to avoid duplicates, and it only keeps tests that will run reliably in a regression suite (no hard-coded names, IDs, or other data tied to a single PR). Labels are applied automatically on promotion — you do not manage them in the prompt.

### Example prompts

Use these as inspiration — copy, combine, or adapt them:

**Conservative regression maintenance (default):**

```
Now that this pull request is merged, update tests the merge changed, promote the
main-flow candidate tests that passed during review, and convert clearly stale tests to
draft. Skip edge cases. Never keep tests with hard-coded PR-specific data.
```

**Only cover normal end-user flows, not admin tooling:**

```
Maintain tests for this merge, but only for flows a normal end user would hit. Do not
promote or create tests that exercise admin-only screens or internal tooling.
```

**Add to the Smoke plan when criteria are met:**

```
Keep the regression suite healthy for this merge. If a promoted test covers a critical,
high-traffic flow (login, checkout, signup) and is stable, also add it to the "Smoke"
test plan.
```

**Be more aggressive about removing stale tests:**

```
After this merge, be proactive about converting tests to draft when the PR removed or
replaced the feature they cover. Explain your reasoning for each one you draft.
```

**Do something other than test maintenance:**

```
When this PR merges, open a Linear issue in the QA team summarizing the user-facing
changes and any test gaps you noticed during review. Do not modify any tests.
```

**Post a Slack summary after merge:**

Requires [Slack tools in chat](/integrations/slack#slack-tools-in-chat) to be enabled
for the project.

```
When this PR merges, post a short Slack summary of the user-facing changes to the
project's default Slack channel. Do not modify any tests.
```

<Note>
  The post-merge agent runs before the internal review retrospective, which
  always runs afterward regardless of whether the post-merge agent is enabled,
  skipped, or fails. Non-merged (closed) PRs skip the post-merge agent and go
  straight to the retrospective.
</Note>

### Post-merge via API or CI

When you trigger a merge review with [`POST /v1/chat/change-review`](/api-reference/chat/start-change-review-chat) (`mode: "pr"`), you can pass optional `postMerge` options. Those options are stored on the review conversation and **override** the GitHub App integration settings for that PR only when it merges.

```bash theme={null}
curl -sSf -X POST "https://api.qa.tech/v1/chat/change-review" \
  -H "Authorization: Bearer $QATECH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "mode": "pr",
    "prUrl": "https://github.com/acme/web/pull/42",
    "vcsProviderId": "github",
    "applicationOverrides": [
      { "id": "app_...", "environmentId": "env_..." }
    ],
    "postMerge": {
      "enabled": true,
      "prompt": "After merge, promote stable main-flow review tests and draft clearly stale ones."
    }
  }'
```

| Field               | Required                       | Description                                                                                      |
| :------------------ | :----------------------------- | :----------------------------------------------------------------------------------------------- |
| `postMerge.enabled` | Yes (when `postMerge` is sent) | `true` runs the post-merge agent after merge; `false` skips it even if the integration has it on |
| `postMerge.prompt`  | No                             | Full goal for the agent. Omit or leave empty to use the default maintenance prompt               |

Omit `postMerge` entirely to use the GitHub App integration toggle and prompt. From CI, pass the same JSON body to the change-review API (or wrap it in your own workflow step).

Promoted tests are labelled `auto-added` for provenance today; a durable, structured link back to the originating pull request, and per-test-plan opt-in controls for automatic additions, are planned follow-ups.

## Test run implementation patterns

### Run on pull requests

```yaml theme={null}
name: PR Tests
on:
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: QAdottech/run-action@v4
        with:
          project_short_id: 'proj_abc123'
          api_token: ${{ secrets.QATECH_API_TOKEN }}
          test_plan_short_id: 'smoke-tests'
          blocking: true
```

### Test preview deployments

```yaml theme={null}
name: Test Preview
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  deploy:
    runs-on: ubuntu-latest
    outputs:
      preview_url: ${{ steps.deploy.outputs.url }}
    steps:
      - name: Deploy to Vercel
        id: deploy
        run: |
          # Your deployment logic
          echo "url=https://preview-${{ github.event.pull_request.number }}.vercel.app" >> $GITHUB_OUTPUT

  test:
    needs: deploy
    runs-on: ubuntu-latest
    steps:
      - uses: QAdottech/run-action@v4
        with:
          project_short_id: 'proj_abc123'
          api_token: ${{ secrets.QATECH_API_TOKEN }}
          test_plan_short_id: 'regression-suite'
          blocking: true
          applications_config: |
            {
              "applications": {
                "frontend-app": {
                  "environment": {
                    "url": "${{ needs.deploy.outputs.preview_url }}",
                    "name": "PR-${{ github.event.pull_request.number }}",
                    "customHeaders": [
                      {
                        "domains": ["*.vercel.app"],
                        "headers": {
                          "x-vercel-protection-bypass": "${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }}",
                          "x-vercel-set-bypass-cookie": "true"
                        }
                      }
                    ]
                  }
                }
              }
            }
```

This pattern passes the preview URL into the Test Run Action. It does not create GitHub deployment records. If you also want the [GitHub App](/configuration/github-app) to pick up the same preview for automatic PR reviews, add the steps in [GitHub Deployments](/configuration/github-deployments).

### Test a mobile PR build

Native apps have no preview URL. Build the APK or simulator `.app` in the workflow, [upload it](/api-reference/application-builds), then pass `applicationBuildShortId` in `applications_config` instead of `url`. Full workflows (test plan and change review) are in [PR Testing for Mobile Apps](/pr-testing/agents/mobile).

### Persist environment custom headers

Pass `customHeaders` on any environment in `applications_config` to persist host-pattern header rules (auth and protection bypass). The Test Run Action and Change Review Action both accept this field. The [preview deployment example](#test-preview-deployments) shows it in a full workflow.

```json theme={null}
{
  "url": "https://preview.example.com",
  "name": "PR-123",
  "customHeaders": [
    {
      "domains": ["*.preview.example.com"],
      "headers": {
        "x-vercel-protection-bypass": "YOUR_SECRET",
        "x-vercel-set-bypass-cookie": "true"
      }
    }
  ]
}
```

`customHeaders` works with `url`, `shortId`, or `applicationBuildShortId`. Omit the field to leave stored headers unchanged. Pass `[]` to clear them. The [Start Run API](/api-reference/runs/start-test-run) uses the same `customHeaders` shape on `applications[].environment` (array of application objects, not a map). See [Environment custom headers](/core-concepts/applications-and-environments#custom-headers).

### Scheduled testing

```yaml theme={null}
name: Nightly Tests
on:
  schedule:
    - cron: '0 2 * * *' # Daily at 2 AM UTC

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: QAdottech/run-action@v4
        with:
          project_short_id: 'proj_abc123'
          api_token: ${{ secrets.QATECH_API_TOKEN }}
          test_plan_short_id: 'full-regression'
          blocking: false
```

### Use action outputs

```yaml theme={null}
- uses: QAdottech/run-action@v4
  id: qatech
  with:
    project_short_id: 'proj_abc123'
    api_token: ${{ secrets.QATECH_API_TOKEN }}
    blocking: true

- name: Check Results
  if: steps.qatech.outputs.run_result == 'FAILED'
  run: echo "Tests failed! See ${{ steps.qatech.outputs.run_url }}"
```