> ## Documentation Index > Fetch the complete documentation index at: https://docs.qa.tech/llms.txt > Use this file to discover all available pages before exploring further. # GitHub pull request testing > Review a GitHub pull request on a preview URL, after merge, or against a mobile build. Use this when the change is a GitHub pull request. To run a test plan you already have, use [GitHub regression testing](/regression-testing/github). ## Set it up Automatic review on every pull request. Needs a preview URL from a deployment or a `@qa.tech` comment. Start the review from a workflow, after a deploy job or on a label. The App must still be installed. No preview environment. Review the merged pull request against staging or production. Upload the simulator or emulator build and pin the review to it. To publish preview URLs from Actions, create [GitHub Deployments](/configuration/github-deployments). A preview behind Vercel deployment protection needs the [Vercel preview bypass](/configuration/vercel-preview-protection). ## What works on GitHub | Capability | GitHub | | :------------------------------------------- | :--------------------------------------------------------------------------------------------------------------- | | Review on a preview URL, before merge | Yes. The GitHub App, or the Change Review Action when the review must wait for your deploy job. | | Review after merge, with no preview URL | Yes. A workflow on the default branch runs the review against the deployed environment. | | Review a mobile build for that pull request | Yes. Upload the binary from Actions and pin the review to it. The App alone tests the default build. | | Run a regression test plan | Not from this guide. The Test Run Action does that. See [GitHub regression testing](/regression-testing/github). | | Block the workflow until the review finishes | Yes on the Change Review Action. The automatic GitHub App posts the review and does not fail the workflow. | ## What this does not do * The GitHub App does not test the pull request's mobile binary. CI has to upload that build. * The automatic App does not fail the GitHub check's workflow job. Use the Change Review Action when the job should wait. * This setup does not run a test plan you picked. That is [GitHub regression testing](/regression-testing/github). ## GitHub App The App starts a review when a pull request opens or updates, and when someone comments `@qa.tech`. Install and repository settings are on [GitHub App](/configuration/github-app). What a review does, how it selects tests, and when it creates tests are the same on every CI. See [How a review works](/pr-testing/overview#how-a-review-works). ### What the GitHub App posts The GitHub App needs write permissions to communicate test results back to your PR. Here is what it posts and when: | Operation | When | Description | | :--------------------------- | :-------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **PR comment (in progress)** | When the agent starts testing | Comment with link to the QA.tech conversation and details (branch, commit SHA, event type). Lets you track progress. | | **PR comment (quota limit)** | When the organization is out of credits | Single comment with upgrade link. No tests run. | | **PR review** | After tests complete | Native GitHub review with verdict (approve/request changes/comment), summary, test results table, and evaluation details. Replaces any pending review from the bot. | | **Reaction (eyes emoji)** | When processing starts | Added to the triggering comment or PR to acknowledge it has been seen. | | **Commit status** | During and after test run | GitHub check named **QA.tech / PR Review** on the PR commit. See [PR Review check](#pr-review-check) below for when it appears and what each state means. | If you see permission errors when installing the app, ensure the repository grants write access for pull requests and statuses. The app only writes when tests run or when the organization is out of credits. ### PR Review check QA.tech registers a GitHub check run named **QA.tech / PR Review** on PR commits. You can require it in branch protection as a merge gate. Whether a check appears on PR open or sync depends on the [integration settings](/configuration/github-app#integration-settings) and how the review was triggered. | Situation | Check created on PR open/sync? | What you see | | :--------------------------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------- | | **Auto-run on PRs** enabled (default) | Yes | `in progress` while the agent reviews, then a final verdict (approve, request changes, or comment) | | **Auto-run on PRs** disabled | No | No check until you explicitly trigger a review (see below) | | Draft PR and **Include draft PRs** disabled | Yes (skipped) | Skipped check titled **Draft PR** - auto-review does not run on drafts until the PR is marked ready for review, or until you comment `@qa.tech` | | Comment `@qa.tech` on the PR | Yes (when the review starts) | `in progress`, then final verdict | | [Change Review Action](/configuration/github-actions#change-review-action) or [Start change review chat API](/api-reference/chat/start-change-review-chat) | Yes (when triggered) | `in progress`, then final verdict. Runs even when **Auto-run on PRs** is off | **Auto-run on PRs** controls only the automatic PR-open trigger. It does not block explicit triggers: `@qa.tech` comments, the Change Review Action, and the change-review API always start a review (and create or update the check) when invoked. When **Auto-run on PRs** is off, QA.tech does **not** post a skipped check that says auto-review is disabled. The PR simply has no QA.tech check until you trigger a review manually or from CI. ### Triggering a PR Review #### Automatic trigger (simple) With **Run automatically on PRs** enabled, QA.tech starts a review when a PR is opened or updated and its preview deployment is ready. This is the simplest option once a repository is configured. Automatic triggering works best when you have a **single application** deployed via [GitHub Deployments](/configuration/github-deployments) (or a provider that creates them, such as Netlify or Vercel) and you don't need extra control over when reviews run. If you have multiple applications per PR, don't use deployment-based previews, or need to gate reviews on labels, paths, or a specific deploy job, use the [Change Review Action](#change-review-action) instead. #### Manual trigger with an `@qa.tech` comment You can trigger a review at any time by commenting `@qa.tech` on the pull request. This works on the main PR conversation and on inline code review comments, even when **Run automatically on PRs** is turned off or a previous review was abandoned. The mention is case-insensitive (`@QA.tech`, `@qatech`, and similar variants all match). When QA.tech picks up the comment, it reacts with an eyes emoji to acknowledge it. | Comment | What happens | | :------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------ | | `@qa.tech` | Re-runs the review. QA.tech waits for a ready preview deployment on the PR's latest commit, then starts automatically. | | `@qa.tech test the checkout flow` | Runs a focused review. Any instructions after the mention are passed to the agent to steer what it tests. | | `@qa.tech https://preview.example.com` | Runs immediately against the URL in the comment, which is treated as the preview deployment to test (skips waiting for a deployment). | Without a URL in the comment, QA.tech waits for a ready preview deployment on the PR's latest commit. It posts a short "waiting for a preview deployment" comment and starts the review automatically once one is ready. Include a preview URL in the comment to start immediately against that environment. #### Re-run from the GitHub checks UI Every review posts a status check on the PR commit. You can re-trigger a review by using GitHub's **Re-run** button on the QA.tech check, the same way you re-run any other CI check. ## Change Review Action Start the review from a workflow when it must wait for a deploy job, a label, or a preview URL you pass in. It runs the same review agent and posts the same native PR review as the App. Turn off **Run automatically on PRs** in **Settings → Integrations → GitHub App** when the workflow drives the review, so each pull request is not reviewed twice. Action inputs are in [Change Review Action](/configuration/github-actions#change-review-action). A review with no preview environment is [After merge](#after-merge). A mobile binary is [Mobile build](#mobile-build). ### Change review implementation patterns #### Block the workflow until the review finishes By default the action returns as soon as the chat conversation is created and the agent starts working. Set `blocking: true` to wait for the agent to finish (and post its native PR review) before the workflow step completes. The step fails when the review ends in `FAILED` or `CANCELLED`, which makes it suitable as a deployment gate. ```yaml theme={null} - uses: QAdottech/run-action/change-review@v4 id: review with: project_short_id: 'proj_abc123' api_token: ${{ secrets.QATECH_API_TOKEN }} blocking: true applications_config: | { "applications": { "YOUR_APP_SHORT_ID": { "environment": { "url": "https://preview-${{ github.event.number }}.example.com" } } } } - name: Log conversation if: always() run: | echo "Status: ${{ steps.review.outputs.chat_status }}" echo "Conversation URL: ${{ steps.review.outputs.chat_url }}" echo "Final assistant message: ${{ steps.review.outputs.chat_response }}" ``` The native review is posted to the PR by the agent itself. `chat_response` is the agent's final assistant text in the chat (which may summarize what it did but is not the GitHub review body); use it for logging or further automation, not as a replacement for the PR review. #### Route multiple applications to per-PR preview URLs The most common reason to reach for this action over the GitHub App's automatic trigger: projects with more than one application per PR (for example a frontend, a backend, and an admin dashboard). Pass one entry per application so the agent knows which preview URL to use for each. Each entry must include an `environment` with at least one of `url`, `shortId`, or `applicationBuildShortId`. ```yaml theme={null} - uses: QAdottech/run-action/change-review@v4 with: project_short_id: 'proj_abc123' api_token: ${{ secrets.QATECH_API_TOKEN }} applications_config: | { "applications": { "app_frontend": { "environment": { "url": "https://preview-${{ github.event.number }}-web.example.com", "name": "PR-${{ github.event.number }}" } }, "app_backend": { "environment": { "url": "https://preview-${{ github.event.number }}-api.example.com", "name": "PR-${{ github.event.number }}" } } } } ``` #### Override the device preset Pass `devicePresetShortId` alongside `environment` to run the review against a specific [device preset](/test-features/device-presets): ```yaml theme={null} applications_config: | { "applications": { "app_frontend": { "environment": { "url": "https://preview.example.com" }, "devicePresetShortId": "preset_mobile_abc123" } } } ``` #### Add free-form context Use the `context` input to guide the review with PR-specific or repo-wide instructions. You can also generate this context automatically with an AI agent — see [PR Testing best practices](/best-practices/pr-testing#pass-generated-context-to-the-change-review-action). ```yaml theme={null} - uses: QAdottech/run-action/change-review@v4 with: project_short_id: 'proj_abc123' api_token: ${{ secrets.QATECH_API_TOKEN }} context: | Focus on the new Apple Pay flow. Test login is shared with the existing checkout test and uses test-user@example.com. applications_config: | { "applications": { "YOUR_APP_SHORT_ID": { "environment": { "url": "https://preview.example.com" } } } } ``` #### Review a different PR Set `pr_url` when the workflow runs outside a `pull_request` event (for example on `workflow_dispatch`): ```yaml theme={null} on: workflow_dispatch: inputs: pr_url: description: PR to review required: true jobs: review: runs-on: ubuntu-latest steps: - uses: QAdottech/run-action/change-review@v4 with: project_short_id: 'proj_abc123' api_token: ${{ secrets.QATECH_API_TOKEN }} pr_url: ${{ inputs.pr_url }} applications_config: | { "applications": { "YOUR_APP_SHORT_ID": { "environment": { "shortId": "env_staging" } } } } ``` ## After merge No preview environment. Review the merged pull request against staging or production. If you do not have per-PR preview environments, run the change review **after** merging to your main branch. The workflow deploys to staging/production, extracts the PR number from the merge commit, and reviews the merged PR's changes against the deployed environment. Results are posted back on the merged PR as a comment. ```yaml theme={null} name: Deploy and Review on: push: branches: [main] jobs: deploy: runs-on: ubuntu-latest steps: # Your existing deploy steps here - run: echo "Deploy to staging..." change-review: needs: deploy runs-on: ubuntu-latest steps: - name: Extract PR number from merge commit id: pr env: COMMIT_MSG: ${{ github.event.head_commit.message }} run: | PR_NUMBER=$(echo "$COMMIT_MSG" | grep -oE '\(#[0-9]+\)' | tail -1 | grep -oE '[0-9]+') if [ -n "$PR_NUMBER" ]; then echo "url=https://github.com/${{ github.repository }}/pull/$PR_NUMBER" >> "$GITHUB_OUTPUT" echo "found=true" >> "$GITHUB_OUTPUT" fi - name: Start QA.tech change review if: steps.pr.outputs.found == 'true' uses: QAdottech/run-action/change-review@v4 with: project_short_id: 'proj_abc123' api_token: ${{ secrets.QATECH_API_TOKEN }} pr_url: ${{ steps.pr.outputs.url }} context: 'This PR has already been merged and deployed. Test the changes and report any issues found as a comment.' applications_config: | { "applications": { "YOUR_APP_SHORT_ID": { "environment": { "shortId": "env_staging" } } } } ``` Notes: * Pass `environment.shortId` for the existing staging environment; pass `url` only when staging is not already an environment in the project. * The PR number is extracted from GitHub's default merge/squash commit message format (`(#123)`). * Direct pushes to `main` without a PR are skipped automatically (the review step is gated on `steps.pr.outputs.found`). * Results are posted back on the merged PR as a comment, since the PR is already closed. ## Mobile build Store `QATECH_API_TOKEN` under **Settings → Secrets and variables → Actions**. Upload the build first. The shared upload script is on [Mobile](/pr-testing/agents/mobile#upload-the-pr-build). ### Run a change review on the PR Use this when you want the review agent to select tests from the diff and post a native GitHub review. Install the GitHub App at the organization level with access to the repository. Turn off **Auto-run on PRs** for that repo in **Settings → Integrations → GitHub App** so QA.tech does not also start a review against the default environment. ```yaml theme={null} name: QA.tech mobile change review on: pull_request: jobs: review: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Build APK run: ./gradlew assembleDebug - name: Upload build to QA.tech id: upload env: QATECH_API_TOKEN: ${{ secrets.QATECH_API_TOKEN }} APK_PATH: app/build/outputs/apk/debug/app-debug.apk run: | APP_ID="app_gXeBl2" FILE_NAME=$(basename "$APK_PATH") UPLOAD_RESPONSE=$(curl -sSf -X POST "https://api.qa.tech/v1/applications/$APP_ID/builds/upload-url" \ -H "Authorization: Bearer $QATECH_API_TOKEN" \ -H "Content-Type: application/json" \ -d "{\"fileName\": \"$FILE_NAME\"}") UPLOAD_URL=$(echo "$UPLOAD_RESPONSE" | jq -r '.uploadUrl') BUILD_TOKEN=$(echo "$UPLOAD_RESPONSE" | jq -r '.buildToken') curl -sSf -X PUT "$UPLOAD_URL" \ --upload-file "$APK_PATH" \ -H "Content-Type: application/octet-stream" BUILD_RESPONSE=$(curl -sSf -X POST "https://api.qa.tech/v1/applications/$APP_ID/builds" \ -H "Authorization: Bearer $QATECH_API_TOKEN" \ -H "Content-Type: application/json" \ -d "{\"platform\": \"android\", \"buildToken\": \"$BUILD_TOKEN\"}") echo "build_short_id=$(echo "$BUILD_RESPONSE" | jq -r '.applicationBuildShortId')" >> "$GITHUB_OUTPUT" - uses: QAdottech/run-action/change-review@v4 with: project_short_id: 'proj_abc123' api_token: ${{ secrets.QATECH_API_TOKEN }} applications_config: | { "applications": { "app_gXeBl2": { "environment": { "applicationBuildShortId": "${{ steps.upload.outputs.build_short_id }}" } } } } ``` Without the GitHub App, the action can create a chat but never comment on the PR. Install it at the organization level with access to the repository. The Change Review Action reads the PR URL from `github.event.pull_request.html_url` on `pull_request` events. Set `blocking: true` if the job should wait for the verdict. Inputs and outputs are in [Change Review Action](/configuration/github-actions#change-review-action). Write a PR description that names the user-facing flows to test. The agent uses that context the same way it does for web PRs. See [PR Testing](/best-practices/pr-testing). To run a known test plan against the same build, see [Mobile regression testing](/regression-testing/agents/mobile#run-a-test-plan-on-the-pr). ## Common questions **Can I control which tests run?** The GitHub App selects tests on its own. Steer a run by commenting `@qa.tech ` on the pull request, for example `@qa.tech test the checkout flow`. See [Triggering a PR Review](#triggering-a-pr-review). To choose the tests yourself, run a test plan with [GitHub regression testing](/regression-testing/github). You can use both. **How do I trigger or re-run a review manually?** Comment `@qa.tech` on the pull request. Add instructions to focus the review, or a preview URL to test a specific environment (`@qa.tech https://preview.example.com`). You can also use GitHub's **Re-run** button on the QA.tech check. **My project has more than one application per pull request. What can I do?** Keep the GitHub App installed, turn off **Auto-run on PRs** in **Settings → Integrations → GitHub App**, and start reviews from the [Change Review Action](#change-review-action). Pass one `applications_config` entry per application so each one gets its own preview URL. With auto-run off, the **QA.tech / PR Review** check appears when the action runs. **Can I gate reviews on labels, paths, or a specific deploy job?** Yes. Use the [Change Review Action](#change-review-action) and put the gating in your workflow (`if:` on labels or paths, `needs:` on a deploy job). Turn off **Auto-run on PRs** if only the action should start reviews. **How do I review a native iOS or Android app?** Automatic App reviews look for a preview URL. A mobile pull request is a new binary. Upload the build from CI and pin the review to it. See [Mobile build](#mobile-build). Turn off **Auto-run on PRs** for that repository so the App does not also review the default build.