PR Testing

A change review reads the pull request or merge request, selects existing tests that cover the user-facing changes, creates tests for what is missing, runs them, and posts the result on the request.

When the tests run

Pick the timing that matches where the new code is reachable. These are two options, not two steps.

Before merge

Preview environment. The review runs against that request’s own deployment, before the code reaches your main branch.

After merge

No preview environment. Run the review after the merge lands and the deploy finishes. The agent tests the merged diff against staging or production.

On a preview environment, before merge

The review starts when the request opens or updates and its preview is ready. Your GitHub or GitLab integration can start it on its own, or a CI job can start it after the preview deploys. QA.tech gets the target from one of these:

A deployment your Git host reports, including GitHub Deployments created from Actions
A preview URL in a @qa.tech comment on the request
For a mobile app, the build you upload for that pull request. Mobile reviews use the binary, not a preview URL.
For an API application, the preview base URL

When that preview is behind Vercel deployment protection, set the Vercel preview bypass on the application’s preview defaults. The bypass is not a URL.

GitHub

The GitHub App, or the Change Review Action after your deploy job.

GitLab

Merge request reviews on deploy, comment, or CI.

Other CI

Call the change review API with the preview URL.

After merge, when you have no preview environment

Run the review after the merge lands and the deploy finishes. The agent reads the merged diff and tests against your staging or production environment. The result is posted on the merged request. Add one job to your CI that runs after the deploy job. It reads the request number from the merge commit and starts a change review with:

The URL of the merged pull request or merge request
The QA.tech environment to test, by its environment short ID (for example env_abc123) or by URL

With an environment short ID, the agent tests the URL saved on that environment in QA.tech. Before you add the job, check that:

Your repository is connected to QA.tech, so the agent can read the diff and post the result.
Your staging environment exists in Settings → Applications & Envs, and you have its environment short ID.
QA.tech can reach staging. If staging is private or behind a VPN, see Environment access.

GitHub

A workflow on your default branch, after the deploy job.

GitLab

A pipeline job on your default branch, after the deploy job.

Other CI

Call the change review API after your deploy finishes.

A post-merge change review tests the merged code. The post-merge agent is a later step on GitHub: it can promote tests from that review into your regression suite. It does not replace the review.

Set it up from your editor with the MCP

If your coding agent (Cursor, Claude Code, Codex) has the QA.tech MCP server connected, it can read your CI, find your IDs, ask QA.tech to create a missing staging environment, and add the CI job. Open your repository in the editor and paste this prompt:

Set up QA.tech change reviews in this repository with the
QA.tech MCP.

1. Read this repository's CI and deploy config. Find which
   CI it uses, whether pull requests get a preview
   deployment, and which branch deploys which environment,
   at what URL.
2. Call list_projects and pick the QA.tech project for this
   app. Call list_applications and
   list_application_environments to find the application
   short ID and the environment whose URL matches the
   deployed environment.
3. If no environment matches, call create_chat with:
   "Create an environment named <name> with URL <url> for
   application <application short ID>, check that QA.tech
   can reach it, and tell me whether <repository URL> is
   connected to QA.tech." Poll get_chat_conversation until
   its status is completed, then list the environments
   again to get the environment short ID.
4. Read the matching guide: docs.qa.tech/pr-testing/github,
   docs.qa.tech/pr-testing/gitlab, or
   docs.qa.tech/pr-testing/api for other CI. If pull
   requests get previews, add the before-merge review. If
   not, add the after-merge review job after the deploy job,
   with the environment short ID.
5. Tell me which CI secret or variable to add for the
   QA.tech API token, and anything I still have to do in
   QA.tech, such as connecting the repository or
   allowlisting IP addresses.

Review the CI change before you merge it.

How a review works

This applies to every CI. The CI guides below cover only how the review starts.

What a review does

Feature	Description
✅ Intelligent Test Selection	AI semantically matches PR changes to affected user flows and relevant tests
✅ Risk-Based Coverage	Blast radius and test risk determine how deeply each affected flow is tested
✅ Gap-Only Test Generation	Creates tests for uncovered paths, with the number and depth based on risk
✅ Persistent Test Suite	Auto-generated tests become permanent regression tests for future PRs
✅ Preview Environment Testing	Tests against PR preview deployments
✅ Approval/Rejection	Posts reviews with pass/fail verdicts
✅ Manual Trigger	Comment `@qa.tech` on a PR to trigger, re-run, or steer a review on demand
❌ Backend-Only Testing	Requires UI access - can’t test microservices without frontend

How a review runs

PR opened/updated
  ↓
1. Classify Changes
   → User-facing? Continue
   → Docs/infra only? Skip testing, post info comment
  ↓
2. Assess Coverage
   → Find relevant existing tests
   → Identify gaps in coverage
  ↓
3. Analyze Risk
   → Map the blast radius and breakable paths
   → Set the depth of testing
  ↓
4. Create Tests (if needed)
   → Generate tests for untested functionality
   → Configure dependencies (e.g., login tests)
  ↓
5. Run Tests
   → Execute against PR preview environment
   → Wait for completion
  ↓
6. Post Review
   → ✅ Approve if all tests pass
   → ❌ Decline if tests fail
   → ℹ️ Informational if untestable

The agent posts reviews with:

Verdict: Pass/fail/unable to verify
What was tested: Description of coverage
Results summary: Patterns and themes
Test details: Table with individual test results

Reviews report test results. They do not include code-quality opinions or implementation suggestions. The agent can use comments from engineers and other bots when it chooses what to test. The posted review does not cite those comments. A bug appears in the review only when a test reproduces it.

Test selection and creation

How tests are selected

The agent uses AI to semantically match PR changes to test goals - not keyword matching. Example: PR changes checkout payment flow

Your project: 645 total tests

Agent analyzes:
├─ "Complete checkout with credit card" → RELEVANT ✓
├─ "Complete checkout with PayPal" → RELEVANT ✓
├─ "Verify payment confirmation email" → RELEVANT ✓
├─ "User profile photo upload" → NOT RELEVANT ✗
└─ "Search products by category" → NOT RELEVANT ✗

Selected: 12 tests covering payment & checkout flows

Selection mechanics:

Selects tests that exercise the changed behavior and affected adjacent flows
Prefers active tests, but can run relevant draft tests
Adds existing smoke coverage for affected user journeys when test risk is high
Does not run unrelated tests just because they belong to the same application

How blast radius sets test depth

QA.tech does not use the same test count for every request. After mapping the change to user flows and existing coverage, the agent assesses how much runtime verification the change warrants. This is a test-risk assessment, not a code-quality score. The agent first inventories the user-observable paths the change could break. Depending on the change, these can include:

The intended success path
Empty, invalid, denied, and error states
Interactions with existing filters, sorting, tabs, or pagination
State that should persist after reload or navigation
Adjacent flows, user roles, and repeated surfaces that share the changed behavior

It then weighs three signals:

Signal	What the agent considers
Blast radius	How many distinct user-facing flows or surfaces can be affected. Shared components and behavior repeated across routes increase the radius.
Flow centrality	How critical the affected flow is. Authentication, sign-up, payments, and actions involving user data receive more scrutiny.
Residual coverage	Which affected paths already have passing tests and which success, edge, or regression paths remain uncovered.

The resulting risk level changes the review depth:

Test risk	Coverage response
Low	Focused coverage of a narrow, peripheral, or well-covered change, including applicable edge paths
Medium	Broader success, edge, persistence, combination, and adjacent regression coverage
High	Coverage of every reachable affected path and context, plus existing smoke tests for the primary and adjacent user journeys
Not assessed	Default review depth when the diff or project knowledge is too limited to determine which user flows the change can affect

Every user-facing functional change still gets direct success-path coverage. Diff size alone does not determine risk. A small change to checkout can be high risk because it affects a central flow, while a larger change to an isolated, well-covered settings page can be low risk. A shared authentication change has a wide blast radius when sign-in, sign-up, and account recovery all depend on it, so the agent covers each reachable flow. The review reports its risk level and rationale under the result headline. Risk controls coverage depth; it does not change the pass, fail, or unable-to-verify rules.

When tests are created

Tests are created only when coverage gaps exist, such as when you’re developing a new feature not yet covered by your testing suite.

Writing acceptance criteria? Include test requirements in your Linear/Jira ticket or PR description. The agent reads this context and uses it to create more accurate tests for your new feature. See Recommended Workflow for New Features for details.

The number of new tests depends on the uncovered breakable paths and the test risk. A narrow, well-covered change may need no new tests. A central change with a wide blast radius can require several new tests and existing smoke tests. Example: PR adds Apple Pay to checkout

Agent assesses existing coverage:
├─ "Complete checkout with credit card" ✓ Exists
├─ "Complete checkout with PayPal" ✓ Exists
└─ Apple Pay integration ✗ Gap identified

Decision: Create 1 new test
→ "Complete checkout with Apple Pay"

Created tests:

Stay in your suite labeled ‘ephemeral’
Available for future PRs
Manage in Settings → Test Cases (filter by ‘ephemeral’ to find them)

Example lifecycle:

PR #42: Add Apple Pay
├─ Agent creates "Complete checkout with Apple Pay"
├─ Test runs on PR #42 ✅
└─ Test persists with 'ephemeral' label

PR #58: Refactor checkout UI
├─ Agent finds existing test
├─ Runs it (no new test created) ✅
└─ Your suite now protects against Apple Pay regressions

The agent prevents duplicates by reading all existing tests first and using semantic deduplication. If one slips through, simply delete it in the UI. Self-limiting: As your test suite grows, fewer tests are created automatically - better coverage means fewer gaps. Most PRs (bug fixes, refactors) create zero new tests.

Mobile and Responsive Testing: When a PR mentions mobile, tablet, or responsive design changes, the AI agent may automatically test with appropriate device presets. The agent detects keywords like “mobile”, “tablet”, “responsive”, or “Safari mobile” and can override device presets when creating or running tests. See Device Presets for configuring device settings.

Common questions

Will generated tests clutter my test suite? No. The agent only creates tests for coverage gaps. Most bug fixes and refactors create none, and a large feature usually adds a few focused tests. Better coverage means fewer gaps, so fewer tests are generated over time. The agent created tests I don’t need. What can I do?

Add review context with specific guidelines in the integration settings.
Update existing tests so they cover the functionality.
The agent follows the patterns in your existing tests.

How do roles and feature flags work? A role is another test account. Save each account as a config and describe which role it is. The agent logs in with that account on the same preview. For a feature flag, match how the product reads it. When the flag is a request header, set it on a device preset or on environment custom headers. When the flag is a control in the product or an API call, say when to turn it on in a PR Review Rule. Does the agent delete the data it creates? No. Add a later test that removes the records, or an API test that does the cleanup. A PR Review Rule or Agent Rule can tell the agent to clean up before it finishes. A preview environment that is replaced on each deploy does not need this. What happens when the organization is out of credits? QA.tech posts one comment on the request with an Add credits link. When it has found issues on more than 1% of the reviewed pull requests, the comment also says how many pull requests it has reviewed and issues it has found in your organization so far. No tests run, and no review is posted.

Pick your CI

Each guide is only the pull request setup. The same CI names under Regression Testing run a test plan instead.

GitHub

App, Change Review Action, after merge, or mobile.

GitLab

Merge request reviews on deploy, comment, CI, or API.

Bitrise

Review the mobile build Bitrise produced.

fastlane

Review the mobile build a fastlane lane produced.

Envoyer

Not supported. See what to use instead.

API

Bitbucket, Jenkins, CircleCI, or any HTTP client.

Writing PR descriptions

Give the review the context it needs.

Give the review enough context

The agent reads the request description, the diff, and linked tickets. A description that states what changed, why, and what to test produces a more relevant review. See Writing PR descriptions.

Agents

Choose your CI

PR Testing

When the tests run

Before merge

After merge

On a preview environment, before merge

GitHub

GitLab

Other CI

After merge, when you have no preview environment

GitHub

GitLab

Other CI

Set it up from your editor with the MCP

How a review works

What a review does

How a review runs

Test selection and creation

How tests are selected

How blast radius sets test depth

When tests are created

Common questions

Pick your CI

GitHub

GitLab

Bitrise

fastlane

Envoyer

API

Writing PR descriptions

Give the review enough context

​When the tests run

Before merge

After merge

​On a preview environment, before merge

GitHub

GitLab

Other CI

​After merge, when you have no preview environment

GitHub

GitLab

Other CI

​Set it up from your editor with the MCP

​How a review works

​What a review does

​How a review runs

​Test selection and creation

​How tests are selected

​How blast radius sets test depth

​When tests are created

​Common questions

​Pick your CI

GitHub

GitLab

Bitrise

fastlane

Envoyer

API

Writing PR descriptions

​Give the review enough context

When the tests run

On a preview environment, before merge

After merge, when you have no preview environment

Set it up from your editor with the MCP

How a review works

What a review does

How a review runs

Test selection and creation

How tests are selected

How blast radius sets test depth

When tests are created

Common questions

Pick your CI

Give the review enough context