Set it up
GitHub App
Automatic review on every pull request. Needs a preview URL from a
deployment or a
@qa.tech comment.Change Review Action
Start the review from a workflow, after a deploy job or on a label. The App
must still be installed.
After merge
No preview environment. Review the merged pull request against staging or
production.
Mobile build
Upload the simulator or emulator build and pin the review to it.
What works on GitHub
What this does not do
- The GitHub App does not test the pull request’s mobile binary. CI has to upload that build.
- The automatic App does not fail the GitHub check’s workflow job. Use the Change Review Action when the job should wait.
- This setup does not run a test plan you picked. That is GitHub regression testing.
GitHub App
The App starts a review when a pull request opens or updates, and when someone comments@qa.tech. Install and repository settings are on GitHub App.
What a review does, how it selects tests, and when it creates tests are the same on every CI. See How a review works.
What the GitHub App posts
The GitHub App needs write permissions to communicate test results back to your PR. Here is what it posts and when:PR Review check
QA.tech registers a GitHub check run named QA.tech / PR Review on PR commits. You can require it in branch protection as a merge gate. Whether a check appears on PR open or sync depends on the integration settings and how the review was triggered.Auto-run on PRs controls only the automatic PR-open trigger. It does not block explicit triggers:
@qa.tech comments, the Change Review Action, and the change-review API always start a review (and create or update the check) when invoked.When Auto-run on PRs is off, QA.tech does not post a skipped check that says auto-review is disabled. The PR simply has no QA.tech check until you trigger a review manually or from CI.Triggering a PR Review
Automatic trigger (simple)
With Run automatically on PRs enabled, QA.tech starts a review when a PR is opened or updated and its preview deployment is ready. This is the simplest option once a repository is configured.Automatic triggering works best when you have a single application
deployed via GitHub Deployments (or a
provider that creates them, such as Netlify or Vercel) and you don’t need
extra control over when reviews run. If you have multiple applications per PR,
don’t use deployment-based previews, or need to gate reviews on labels, paths,
or a specific deploy job, use the Change Review
Action instead.
Manual trigger with an @qa.tech comment
You can trigger a review at any time by commenting @qa.tech on the pull request. This works on the main PR conversation and on inline code review comments, even when Run automatically on PRs is turned off or a previous review was abandoned.
The mention is case-insensitive (@QA.tech, @qatech, and similar variants all match). When QA.tech picks up the comment, it reacts with an eyes emoji to acknowledge it.
Without a URL in the comment, QA.tech waits for a ready preview deployment on
the PR’s latest commit. It posts a short “waiting for a preview deployment”
comment and starts the review automatically once one is ready. Include a
preview URL in the comment to start immediately against that environment.
Re-run from the GitHub checks UI
Every review posts a status check on the PR commit. You can re-trigger a review by using GitHub’s Re-run button on the QA.tech check, the same way you re-run any other CI check.Change Review Action
Start the review from a workflow when it must wait for a deploy job, a label, or a preview URL you pass in. It runs the same review agent and posts the same native PR review as the App. Turn off Run automatically on PRs in Settings → Integrations → GitHub App when the workflow drives the review, so each pull request is not reviewed twice. Action inputs are in Change Review Action. A review with no preview environment is After merge. A mobile binary is Mobile build.Change review implementation patterns
Block the workflow until the review finishes
By default the action returns as soon as the chat conversation is created and the agent starts working. Setblocking: true to wait for the agent to finish (and post its native PR review) before the workflow step completes. The step fails when the review ends in FAILED or CANCELLED, which makes it suitable as a deployment gate.
chat_response is the agent’s final assistant text in the chat (which may summarize what it did but is not the GitHub review body); use it for logging or further automation, not as a replacement for the PR review.
Route multiple applications to per-PR preview URLs
The most common reason to reach for this action over the GitHub App’s automatic trigger: projects with more than one application per PR (for example a frontend, a backend, and an admin dashboard). Pass one entry per application so the agent knows which preview URL to use for each. Each entry must include anenvironment with at least one of url, shortId, or applicationBuildShortId.
Override the device preset
PassdevicePresetShortId alongside environment to run the review against a specific device preset:
Add free-form context
Use thecontext input to guide the review with PR-specific or repo-wide instructions. You can also generate this context automatically with an AI agent — see PR Testing best practices.
Review a different PR
Setpr_url when the workflow runs outside a pull_request event (for example on workflow_dispatch):
After merge
No preview environment. Review the merged pull request against staging or production. If you do not have per-PR preview environments, run the change review after merging to your main branch. The workflow deploys to staging/production, extracts the PR number from the merge commit, and reviews the merged PR’s changes against the deployed environment. Results are posted back on the merged PR as a comment.- Pass
environment.shortIdfor the existing staging environment; passurlonly when staging is not already an environment in the project. - The PR number is extracted from GitHub’s default merge/squash commit message format (
(#123)). - Direct pushes to
mainwithout a PR are skipped automatically (the review step is gated onsteps.pr.outputs.found). - Results are posted back on the merged PR as a comment, since the PR is already closed.
Mobile build
StoreQATECH_API_TOKEN under Settings → Secrets and variables → Actions.
Upload the build first. The shared upload script is on Mobile.
Run a change review on the PR
Use this when you want the review agent to select tests from the diff and post a native GitHub review. Install the GitHub App at the organization level with access to the repository. Turn off Auto-run on PRs for that repo in Settings → Integrations → GitHub App so QA.tech does not also start a review against the default environment.github.event.pull_request.html_url on pull_request events. Set blocking: true if the job should wait for the verdict. Inputs and outputs are in Change Review Action.
Write a PR description that names the user-facing flows to test. The agent uses that context the same way it does for web PRs. See PR Testing.
To run a known test plan against the same build, see Mobile regression testing.
Common questions
Can I control which tests run? The GitHub App selects tests on its own. Steer a run by commenting@qa.tech <instructions> on the pull request, for example @qa.tech test the checkout flow. See Triggering a PR Review. To choose the tests yourself, run a test plan with GitHub regression testing. You can use both.
How do I trigger or re-run a review manually?
Comment @qa.tech on the pull request. Add instructions to focus the review, or a preview URL to test a specific environment (@qa.tech https://preview.example.com). You can also use GitHub’s Re-run button on the QA.tech check.
My project has more than one application per pull request. What can I do?
Keep the GitHub App installed, turn off Auto-run on PRs in Settings → Integrations → GitHub App, and start reviews from the Change Review Action. Pass one applications_config entry per application so each one gets its own preview URL. With auto-run off, the QA.tech / PR Review check appears when the action runs.
Can I gate reviews on labels, paths, or a specific deploy job?
Yes. Use the Change Review Action and put the gating in your workflow (if: on labels or paths, needs: on a deploy job). Turn off Auto-run on PRs if only the action should start reviews.
How do I review a native iOS or Android app?
Automatic App reviews look for a preview URL. A mobile pull request is a new binary. Upload the build from CI and pin the review to it. See Mobile build. Turn off Auto-run on PRs for that repository so the App does not also review the default build.