PR Testing

A change review reads the pull request or merge request, selects existing tests that cover the user-facing changes, creates tests for what is missing, runs them, and posts the result on the request.

When the tests run

Pick the timing that matches where the new code is reachable. These are two options, not two steps.

Before merge

Preview environment. The review runs against that request’s own deployment, before the code reaches your main branch.

After merge

No preview environment. Run the review after the merge lands and the deploy finishes. The agent tests the merged diff against staging or production.

On a preview environment, before merge

QA.tech gets the target from one of these:

A deployment your Git host reports, including GitHub Deployments created from Actions
A preview URL in a @qa.tech comment on the request
For a mobile app, the build you upload for that pull request. Mobile reviews use the binary, not a preview URL.
For an API application, the preview base URL

When that preview is behind Vercel deployment protection, set the Vercel preview bypass on the application’s preview defaults. The bypass is not a URL.

After merge, when you have no preview environment

Run the review after the merge lands and the deploy finishes. The agent reads the merged diff and tests against staging or production. The result is posted on the merged request.

A post-merge change review tests the merged code. The post-merge agent is a later step on GitHub: it can promote tests from that review into your regression suite. It does not replace the review.

How it works in practice

Here’s what happens when you open a PR that changes your checkout flow: 1. Automatic Detection

PR opened: "Add Apple Pay to checkout"
→ PR Review agent activates

2. Change Classification

Analyzing diff...
├─ checkout.tsx: USER-FACING ✓
├─ payment-methods.ts: USER-FACING ✓
└─ README.md: DOCS ONLY ✗

Classification: USER-FACING changes detected

3. Coverage Assessment

Existing tests found:
├─ "Complete checkout with credit card" ✓
├─ "Complete checkout with PayPal" ✓
└─ Apple Pay integration: NOT COVERED ⚠️

4. Test Generation

Creating new test:
"Complete checkout flow with Apple Pay"
├─ Add item to cart
├─ Navigate to checkout
├─ Select Apple Pay as payment method
├─ Verify payment confirmation
└─ Verify order appears in account

5. Execution & Review

Running against PR preview environment...
✅ All tests passing (3/3)

Posting GitHub review:
"✅ Tests passing - Apple Pay integration verified"

This entire workflow runs autonomously - no human intervention required.

How a review works

This applies to every CI. The CI guides below cover only how the review starts.

What a review does

Feature	Description
✅ Intelligent Test Selection	AI semantically matches PR changes to affected user flows and relevant tests
✅ Risk-Based Coverage	Blast radius and test risk determine how deeply each affected flow is tested
✅ Gap-Only Test Generation	Creates tests for uncovered paths, with the number and depth based on risk
✅ Persistent Test Suite	Auto-generated tests become permanent regression tests for future PRs
✅ Preview Environment Testing	Tests against PR preview deployments
✅ Approval/Rejection	Posts reviews with pass/fail verdicts
✅ Manual Trigger	Comment `@qa.tech` on a PR to trigger, re-run, or steer a review on demand
❌ Backend-Only Testing	Requires UI access - can’t test microservices without frontend

How a review runs

PR opened/updated
  ↓
1. Classify Changes
   → User-facing? Continue
   → Docs/infra only? Skip testing, post info comment
  ↓
2. Assess Coverage
   → Find relevant existing tests
   → Identify gaps in coverage
  ↓
3. Analyze Risk
   → Map the blast radius and breakable paths
   → Set the depth of testing
  ↓
4. Create Tests (if needed)
   → Generate tests for untested functionality
   → Configure dependencies (e.g., login tests)
  ↓
5. Run Tests
   → Execute against PR preview environment
   → Wait for completion
  ↓
6. Post Review
   → ✅ Approve if all tests pass
   → ❌ Decline if tests fail
   → ℹ️ Informational if untestable

The agent posts reviews with:

Verdict: Pass/fail/unable to verify
What was tested: Description of coverage
Results summary: Patterns and themes
Test details: Table with individual test results

Reviews focus on test results only - no code quality opinions, implementation suggestions, or references to other bot comments.

Test selection and creation

How tests are selected

The agent uses AI to semantically match PR changes to test goals - not keyword matching. Example: PR changes checkout payment flow

Your project: 645 total tests

Agent analyzes:
├─ "Complete checkout with credit card" → RELEVANT ✓
├─ "Complete checkout with PayPal" → RELEVANT ✓
├─ "Verify payment confirmation email" → RELEVANT ✓
├─ "User profile photo upload" → NOT RELEVANT ✗
└─ "Search products by category" → NOT RELEVANT ✗

Selected: 12 tests covering payment & checkout flows

Selection mechanics:

Selects tests that exercise the changed behavior and affected adjacent flows
Prefers active tests, but can run relevant draft tests
Adds existing smoke coverage for affected user journeys when test risk is high
Does not run unrelated tests just because they belong to the same application

How blast radius sets test depth

QA.tech does not use the same test count for every request. After mapping the change to user flows and existing coverage, the agent assesses how much runtime verification the change warrants. This is a test-risk assessment, not a code-quality score. The agent first inventories the user-observable paths the change could break. Depending on the change, these can include:

The intended success path
Empty, invalid, denied, and error states
Interactions with existing filters, sorting, tabs, or pagination
State that should persist after reload or navigation
Adjacent flows, user roles, and repeated surfaces that share the changed behavior

It then weighs three signals:

Signal	What the agent considers
Blast radius	How many distinct user-facing flows or surfaces can be affected. Shared components and behavior repeated across routes increase the radius.
Flow centrality	How critical the affected flow is. Authentication, sign-up, payments, and actions involving user data receive more scrutiny.
Residual coverage	Which affected paths already have passing tests and which success, edge, or regression paths remain uncovered.

The resulting risk level changes the review depth:

Test risk	Coverage response
Low	Focused coverage of a narrow, peripheral, or well-covered change, including applicable edge paths
Medium	Broader success, edge, persistence, combination, and adjacent regression coverage
High	Coverage of every reachable affected path and context, plus existing smoke tests for the primary and adjacent user journeys
Not assessed	Default review depth when the diff or project knowledge is too limited to determine which user flows the change can affect

Every user-facing functional change still gets direct success-path coverage. Diff size alone does not determine risk. A small change to checkout can be high risk because it affects a central flow, while a larger change to an isolated, well-covered settings page can be low risk. A shared authentication change has a wide blast radius when sign-in, sign-up, and account recovery all depend on it, so the agent covers each reachable flow. The review reports its risk level and rationale under the result headline. Risk controls coverage depth; it does not change the pass, fail, or unable-to-verify rules.

When tests are created

Tests are created only when coverage gaps exist, such as when you’re developing a new feature not yet covered by your testing suite.

Writing acceptance criteria? Include test requirements in your Linear/Jira ticket or PR description. The agent reads this context and uses it to create more accurate tests for your new feature. See Recommended Workflow for New Features for details.

The number of new tests depends on the uncovered breakable paths and the test risk. A narrow, well-covered change may need no new tests. A central change with a wide blast radius can require several new tests and existing smoke tests. Example: PR adds Apple Pay to checkout

Agent assesses existing coverage:
├─ "Complete checkout with credit card" ✓ Exists
├─ "Complete checkout with PayPal" ✓ Exists
└─ Apple Pay integration ✗ Gap identified

Decision: Create 1 new test
→ "Complete checkout with Apple Pay"

Created tests:

Stay in your suite labeled ‘ephemeral’
Available for future PRs
Manage in Settings → Test Cases (filter by ‘ephemeral’ to find them)

Example lifecycle:

PR #42: Add Apple Pay
├─ Agent creates "Complete checkout with Apple Pay"
├─ Test runs on PR #42 ✅
└─ Test persists with 'ephemeral' label

PR #58: Refactor checkout UI
├─ Agent finds existing test
├─ Runs it (no new test created) ✅
└─ Your suite now protects against Apple Pay regressions

The agent prevents duplicates by reading all existing tests first and using semantic deduplication. If one slips through, simply delete it in the UI. Self-limiting: As your test suite grows, fewer tests are created automatically - better coverage means fewer gaps. Most PRs (bug fixes, refactors) create zero new tests.

Mobile and Responsive Testing: When a PR mentions mobile, tablet, or responsive design changes, the AI agent may automatically test with appropriate device presets. The agent detects keywords like “mobile”, “tablet”, “responsive”, or “Safari mobile” and can override device presets when creating or running tests. See Device Presets for configuring device settings.

Common questions

Will generated tests clutter my test suite? No. The agent only creates tests for coverage gaps. Most bug fixes and refactors create none, and a large feature usually adds a few focused tests. Better coverage means fewer gaps, so fewer tests are generated over time. The agent created tests I don’t need. What can I do?

Add review context with specific guidelines in the integration settings.
Update existing tests so they cover the functionality.
The agent follows the patterns in your existing tests.

What happens when the organization is out of credits? QA.tech posts one comment on the request with an upgrade link. No tests run, and no review is posted.

Pick your CI

Each guide is only the pull request setup. The same CI names under Regression Testing run a test plan instead.

GitHub

App, Change Review Action, after merge, or mobile.

GitLab

Merge request reviews on deploy, comment, CI, or API.

Bitrise

Review the mobile build Bitrise produced.

Envoyer

Not supported. See what to use instead.

API

Bitbucket, Jenkins, CircleCI, or any HTTP client.

Writing PR descriptions

Give the review the context it needs.

Give the review enough context

The agent reads the request description, the diff, and linked tickets. A description that states what changed, why, and what to test produces a more relevant review. See Writing PR descriptions.

Agents

Choose your CI

PR Testing

When the tests run

Before merge

After merge

On a preview environment, before merge

After merge, when you have no preview environment

How it works in practice

How a review works

What a review does

How a review runs

Test selection and creation

How tests are selected

How blast radius sets test depth

When tests are created

Common questions

Pick your CI

GitHub

GitLab

Bitrise

Envoyer

API

Writing PR descriptions

Give the review enough context

​When the tests run

Before merge

After merge

​On a preview environment, before merge

​After merge, when you have no preview environment

​How it works in practice

​How a review works

​What a review does

​How a review runs

​Test selection and creation

​How tests are selected

​How blast radius sets test depth

​When tests are created

​Common questions

​Pick your CI

GitHub

GitLab

Bitrise

Envoyer

API

Writing PR descriptions

​Give the review enough context

When the tests run

On a preview environment, before merge

After merge, when you have no preview environment

How it works in practice

How a review works

What a review does

How a review runs

Test selection and creation

How tests are selected

How blast radius sets test depth

When tests are created

Common questions

Pick your CI

Give the review enough context