> ## Documentation Index
> Fetch the complete documentation index at: https://docs.qa.tech/llms.txt
> Use this file to discover all available pages before exploring further.
# GitHub App for PR Reviews
> Automatic AI-powered pull request testing and reviews
Get autonomous AI-powered test coverage and reviews on every pull request. The QA.tech GitHub App uses **AI exploratory testing** to analyze code changes, create missing tests, and post comprehensive reviews based on test results.
**Integration modes:**
* **AI exploratory** (this page): Automatic AI-powered PR reviews - GitHub only
* **API-driven**: Manual test plan triggers via [GitHub Actions](/configuration/github-actions) or [GitLab CI](/configuration/gitlab)
See [CI/CD Integration Overview](/configuration/ci-cd-integration) to understand both modes and choose the right approach for your needs.
If you use GitLab merge requests, see [GitLab Merge Request Reviews](/configuration/gitlab).
## What It Does
| Feature | Description |
| :-------------------------------- | :----------------------------------------------------------------------------------- |
| ✅ **Intelligent Test Selection** | AI semantically matches PR changes to relevant tests (typically 5-15 tests selected) |
| ✅ **Gap-Only Test Generation** | Creates 1-3 tests only when coverage gaps exist; most PRs create zero new tests |
| ✅ **Persistent Test Suite** | Auto-generated tests become permanent regression tests for future PRs |
| ✅ **Preview Environment Testing** | Tests against PR preview deployments |
| ✅ **Approval/Rejection** | Posts reviews with pass/fail verdicts |
| ✅ **Manual Trigger** | Comment `@qa.tech` on a PR to trigger, re-run, or steer a review on demand |
| ❌ **Backend-Only Testing** | Requires UI access - can't test microservices without frontend |
## How PR Reviews Work
```
PR opened/updated
↓
1. Classify Changes
→ User-facing? Continue
→ Docs/infra only? Skip testing, post info comment
↓
2. Assess Coverage
→ Find relevant existing tests
→ Identify gaps in coverage
↓
3. Create Tests (if needed)
→ Generate tests for untested functionality
→ Configure dependencies (e.g., login tests)
↓
4. Run Tests
→ Execute against PR preview environment
→ Wait for completion
↓
5. Post Review
→ ✅ Approve if all tests pass
→ ❌ Decline if tests fail
→ ℹ️ Informational if untestable
```
The agent posts reviews with:
* **Verdict:** Pass/fail/unable to verify
* **What was tested:** Description of coverage
* **Results summary:** Patterns and themes
* **Test details:** Table with individual test results
**Reviews focus on test results only** - no code quality opinions, implementation suggestions, or references to other bot comments.
## What the GitHub App Writes
The GitHub App needs write permissions to communicate test results back to your PR. Here is what it posts and when:
| Operation | When | Description |
| :--------------------------- | :-------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **PR comment (in progress)** | When the agent starts testing | Comment with link to the QA.tech conversation and details (branch, commit SHA, event type). Lets you track progress. |
| **PR comment (quota limit)** | When the organization is out of credits | Single comment with upgrade link. No tests run. |
| **PR review** | After tests complete | Native GitHub review with verdict (approve/request changes/comment), summary, test results table, and evaluation details. Replaces any pending review from the bot. |
| **Reaction (eyes emoji)** | When processing starts | Added to the triggering comment or PR to acknowledge it has been seen. |
| **Commit status** | During and after test run | GitHub check named **QA.tech / PR Review** on the PR commit. See [PR Review check on GitHub](#pr-review-check-on-github) below for when it appears and what each state means. |
If you see permission errors when installing the app, ensure the repository
grants write access for pull requests and statuses. The app only writes when
tests run or when the organization is out of credits.
## PR Review check on GitHub
QA.tech registers a GitHub check run named **QA.tech / PR Review** on PR commits. You can require it in branch protection as a merge gate. Whether a check appears on PR open or sync depends on your integration settings and how the review was triggered.
| Situation | Check created on PR open/sync? | What you see |
| :--------------------------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------- |
| **Auto-run on PRs** enabled (default) | Yes | `in progress` while the agent reviews, then a final verdict (approve, request changes, or comment) |
| **Auto-run on PRs** disabled | No | No check until you explicitly trigger a review (see below) |
| Draft PR and **Include draft PRs** disabled | Yes (skipped) | Skipped check titled **Draft PR** - auto-review does not run on drafts until the PR is marked ready for review, or until you comment `@qa.tech` |
| Comment `@qa.tech` on the PR | Yes (when the review starts) | `in progress`, then final verdict |
| [Change Review Action](/configuration/github-actions#change-review-action) or [Start change review chat API](/api-reference/chat/start-change-review-chat) | Yes (when triggered) | `in progress`, then final verdict. Runs even when **Auto-run on PRs** is off |
**Auto-run on PRs** controls only the automatic PR-open trigger. It does not block explicit triggers: `@qa.tech` comments, the Change Review Action, and the change-review API always start a review (and create or update the check) when invoked.
When **Auto-run on PRs** is off, QA.tech does **not** post a skipped check that says auto-review is disabled. The PR simply has no QA.tech check until you trigger a review manually or from CI.
### Integration settings
Configure these under **Settings → Integrations → GitHub App** for the repository:
| Setting | Effect on the PR Review check |
| :-------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Auto-run on PRs** | When on, creates the check on PR open/sync (subject to draft rules below). When off, no check on PR open - use `@qa.tech`, the Change Review Action, or the API. |
| **Include draft PRs** | When off (default), draft PRs get a skipped **Draft PR** check instead of an in-progress review. When on, draft PRs are reviewed the same as non-draft PRs (if auto-run is enabled). |
| **Register a commit check** | On by default. When off, QA.tech never posts the **QA.tech / PR Review** check for any trigger (auto-run, `@qa.tech`, the Change Review Action, or the API). Reviews still post the sticky PR comment and appear in the dashboard, but there is no merge gate to require. |
## Test Selection and Creation
### How Tests Are Selected
The agent uses AI to semantically match PR changes to test goals - not keyword matching.
**Example: PR changes checkout payment flow**
```
Your project: 645 total tests
Agent analyzes:
├─ "Complete checkout with credit card" → RELEVANT ✓
├─ "Complete checkout with PayPal" → RELEVANT ✓
├─ "Verify payment confirmation email" → RELEVANT ✓
├─ "User profile photo upload" → NOT RELEVANT ✗
└─ "Search products by category" → NOT RELEVANT ✗
Selected: 12 tests covering payment & checkout flows
```
**Selection mechanics:**
* Runs ALL tests it determines are relevant (no arbitrary limits)
* Only considers tests with `status='enabled'`
* More intelligent than running all tests every time
### When Tests Are Created
Tests are created only when coverage gaps exist, such as when you're developing a new feature not yet covered by your testing suite.
**Writing acceptance criteria?** Include test requirements in your Linear/Jira
ticket or PR description. The agent reads this context and uses it to create
more accurate tests for your new feature. See [Recommended Workflow for New
Features](/best-practices/creating-tests#recommended-workflow-for-new-features)
for details.
| PR Type | Existing Tests Selected | New Tests Created | Total |
| :--------------- | :---------------------- | :---------------------- | :---- |
| Bug fix in login | 3-7 | 0 (already covered) | 3-7 |
| Small feature | 5-10 | 1-2 (fill gaps) | 6-12 |
| Major feature | 15-25 | 3-5 (new functionality) | 18-30 |
| Refactor | 10-20 | 0 (no new behavior) | 10-20 |
| Docs/infra only | 0 | 0 (untestable via UI) | 0 |
**Example: PR adds Apple Pay to checkout**
```
Agent assesses existing coverage:
├─ "Complete checkout with credit card" ✓ Exists
├─ "Complete checkout with PayPal" ✓ Exists
└─ Apple Pay integration ✗ Gap identified
Decision: Create 1 new test
→ "Complete checkout with Apple Pay"
```
**Created tests:**
* Stay in your suite labeled 'ephemeral'
* Available for future PRs
* Manage in Settings → Test Cases (filter by 'ephemeral' to find them)
**Example lifecycle:**
```
PR #42: Add Apple Pay
├─ Agent creates "Complete checkout with Apple Pay"
├─ Test runs on PR #42 ✅
└─ Test persists with 'ephemeral' label
PR #58: Refactor checkout UI
├─ Agent finds existing test
├─ Runs it (no new test created) ✅
└─ Your suite now protects against Apple Pay regressions
```
**The agent prevents duplicates** by reading all existing tests first and using semantic deduplication. If one slips through, simply delete it in the UI.
**Self-limiting:** As your test suite grows, fewer tests are created automatically - better coverage means fewer gaps. Most PRs (bug fixes, refactors) create zero new tests.
**Mobile and Responsive Testing:** When a PR mentions mobile, tablet, or
responsive design changes, the AI agent may automatically test with
appropriate device presets. The agent detects keywords like "mobile",
"tablet", "responsive", or "Safari mobile" and can override device presets
when creating or running tests. See [Device
Presets](/test-features/device-presets) for configuring device settings.
## How to Set It Up
Go to [Settings → Organization → Connections](https://app.qa.tech/current-project/settings/organization/connections) and add the GitHub App connection. Follow the OAuth flow to grant access to your repositories.
Navigate to [Settings →
Integrations](https://app.qa.tech/current-project/settings/integrations?focus=github-app)
and select the repository you want to enable PR reviews for. **PR reviews are
enabled automatically** once you select a repository. **Optional:** Add review
context to guide the agent (e.g., "Focus on security vulnerabilities" or
"Validate accessibility standards"). This appears in the integration settings.
If you use Vercel, Netlify, Render, Railway, or Fly.io, your preview deployments are detected automatically - skip to step 4.
QA.tech automatically creates preview environment records in your project when it detects your preview deployments (from Vercel, Netlify, etc.). Your CI/CD platform handles the actual deployment - QA.tech just tests against the preview URLs. For more information, see [Preview Environments](/core-concepts/applications-and-environments#preview-environments).
**For CircleCI, Jenkins, GitLab CI, or custom CI/CD:**
QA.tech detects preview deployments using GitHub's [Deployments API](https://docs.github.com/en/rest/deployments/deployments). Your CI/CD needs to create deployment records after deploying.
**What QA.tech expects:**
1. GitHub deployment created for the PR commit SHA
2. Deployment status set to `success`
3. Status includes `target_url` field with your preview URL
**How to set this up:**
Your CI/CD makes two GitHub API calls after deployment:
1. [Create deployment](https://docs.github.com/en/rest/deployments/deployments#create-a-deployment) with commit SHA and environment name
2. [Create deployment status](https://docs.github.com/en/rest/deployments/statuses#create-a-deployment-status) with `state: "success"` and `target_url: "https://your-preview-url.com"`
**CI/CD platform docs for environment variables:**
* [CircleCI Environment Variables](https://circleci.com/docs/env-vars/) - Use `$CIRCLE_SHA1`, `$CIRCLE_PROJECT_USERNAME`, `$CIRCLE_PROJECT_REPONAME`
* [GitLab CI/CD Variables](https://docs.gitlab.com/ee/ci/variables/) - Use `$CI_COMMIT_SHA`, `$CI_PROJECT_NAMESPACE`, `$CI_PROJECT_NAME`
* [Jenkins Credentials](https://www.jenkins.io/doc/book/using/using-credentials/) - Use `$GIT_COMMIT` and custom variables
* [GitHub Actions](https://docs.github.com/en/actions/learn-github-actions/variables) - Use `${{ github.sha }}`, `${{ github.repository_owner }}`
Need help? Contact [support@qa.tech](mailto:support@qa.tech) with your CI/CD platform.
If you have multiple Applications (e.g., frontend + backend), map GitHub
deployment environments to the correct QA.tech Applications. **How it works:**
1. Your CI/CD deploys a PR and creates a GitHub environment (e.g., "Preview"
or "pr-123") 2. QA.tech detects the deployment 3. Tests run using the mapped
Application's URL from that environment **Location:** Settings → Integrations
→ GitHub App → Map Environments
Without GitHub deployments, QA.tech can't test against PR-specific URLs and
will use your default environment.
Once configured, the agent automatically runs on PRs:
1. Detects code changes when PRs are opened or updated
2. Determines which tests are relevant
3. Creates new tests for untested functionality
4. Runs all relevant tests against the PR preview
5. Posts a review with approval or decline based on results
## Triggering a PR Review
### Automatic trigger (simple)
With **Run automatically on PRs** enabled, QA.tech starts a review when a PR is opened or updated and its preview deployment is ready. This is the simplest option once a repository is configured.
Automatic triggering works best when you have a **single application**
deployed via **GitHub deployments** (or a provider that creates them, such as
Netlify or Vercel) and you don't need extra control over when reviews run. If
you have multiple applications per PR, don't use deployment-based previews, or
need to gate reviews on labels, paths, or a specific deploy job, use the
[GitHub Action trigger](#github-action-trigger-recommended) instead.
### GitHub Action trigger (recommended)
For the most control, trigger reviews from your CI with the [Change Review Action](/configuration/github-actions#change-review-action). It runs the same review agent and posts the same native PR review as the automatic trigger, but lets you decide exactly when it fires and pass an `applications_config` per PR - so you can route each application to its own preview URL, gate reviews on labels or paths, or order the review after a specific deploy job.
Turn off **Run automatically on PRs** in **Settings → Integrations → GitHub App** when driving reviews from the action, so each PR isn't reviewed twice. See [GitHub Actions](/configuration/github-actions#change-review-action) for setup details.
### Manual trigger with an `@qa.tech` comment
You can trigger a review at any time by commenting `@qa.tech` on the pull request. This works on the main PR conversation and on inline code review comments, even when **Run automatically on PRs** is turned off or a previous review was abandoned.
The mention is case-insensitive (`@QA.tech`, `@qatech`, and similar variants all match). When QA.tech picks up the comment, it reacts with an eyes emoji to acknowledge it.
| Comment | What happens |
| :------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------ |
| `@qa.tech` | Re-runs the review. QA.tech waits for a ready preview deployment on the PR's latest commit, then starts automatically. |
| `@qa.tech test the checkout flow` | Runs a focused review. Any instructions after the mention are passed to the agent to steer what it tests. |
| `@qa.tech https://preview.example.com` | Runs immediately against the URL in the comment, which is treated as the preview deployment to test (skips waiting for a deployment). |
Without a URL in the comment, QA.tech waits for a ready preview deployment on
the PR's latest commit. It posts a short "waiting for a preview deployment"
comment and starts the review automatically once one is ready. Include a
preview URL in the comment to start immediately against that environment.
### Re-run from the GitHub checks UI
Every review posts a status check on the PR commit. You can re-trigger a review by using GitHub's **Re-run** button on the QA.tech check, the same way you re-run any other CI check.
## Common Questions
**Will auto-generated tests pollute my test suite?**
No. The agent only creates tests for coverage gaps - most PRs (bug fixes, refactors) create zero new tests. Even major features typically add 3-5 focused tests. The system is self-limiting: better coverage → fewer gaps → less generation.
**Can I control which tests run?**
The GitHub App selects tests autonomously, but you can steer a run by commenting `@qa.tech ` on the PR (e.g. `@qa.tech test the checkout flow`) - see [Triggering a PR Review](#triggering-a-pr-review). For full manual test selection, use [GitHub Actions](/configuration/github-actions) instead. You can use both: automatic PR reviews + manual deep testing on-demand.
**How do I trigger or re-run a review manually?**
Comment `@qa.tech` on the pull request. Add instructions to focus the review (`@qa.tech test the checkout flow`) or a preview URL to test a specific environment (`@qa.tech https://preview.example.com`). You can also use GitHub's **Re-run** button on the QA.tech check. See [Triggering a PR Review](#triggering-a-pr-review).
**My project has multiple applications per PR and the App's environment mapping isn't a good fit. What can I do?**
Install the GitHub App as usual, then turn off **Auto-run on PRs** in **Settings → Integrations → GitHub App** and drive reviews from CI with the [Change Review Action](/configuration/github-actions#change-review-action). The action runs the same review agent (and posts the same native PR review) but lets you pass an `applications_config` per PR, so you can route each application short ID to its own preview URL without relying on the App's static environment mapping. With auto-run off, no **QA.tech / PR Review** check appears on PR open; the check is created when the action runs.
**Can I gate reviews on labels, paths, or a specific deploy job?**
Yes. Use the [Change Review Action](/configuration/github-actions#change-review-action) instead of the automatic trigger and put the gating logic in your workflow (`if:` conditions on labels or paths, `needs:` on a deploy job, etc.). Disable **Auto-run on PRs** if you want only the action to fire; the action posts the native PR review the same way the App does.
**PR reviews aren't posting - what should I check?**
* Verify GitHub App is installed and has repository access
* Check repository is configured in QA.tech Settings → Integrations
* Ensure PR has user-facing changes (docs/infra-only PRs are skipped)
* If you see 403 Forbidden, check that the app has write permissions for pull requests and statuses
**We ran out of credits - what happens?**
QA.tech posts a single comment on the PR with an upgrade link. No tests run. No review is posted.
**Tests running against wrong URL?**
* Map GitHub environments to Applications in Settings → Integrations → GitHub App
* Verify your CI/CD creates GitHub deployment records
* Check environment names match between GitHub and QA.tech
**Agent created irrelevant tests?**
* Add review context with specific guidelines in integration settings
* Update existing tests to better cover the functionality
* The agent learns from your existing test patterns
## Related Documentation
* **[CI/CD Integration Overview](/configuration/ci-cd-integration)** - Learn about integration modes and capabilities
* **[GitHub Actions](/configuration/github-actions)** - API-driven testing with manual control over test plans
* **[Test Plans](/core-concepts/test-plans)** - Organize tests for API-driven runs