Skip to main content
Regression testing starts with a test plan: the tests you chose, against the environment on that plan. You decide which tests run. A change review is the other workflow. It picks and creates tests for one pull request, and it does not use the test plan as its source.

​
1. Create a test plan

Create the plan before you connect CI. CI only starts it.

Create a test plan

Choose the tests and the environment.

​
2. Choose how to run it

Manual

Open the test plan and select Run Tests, or use qatech run.

Scheduled

Open the test plan, select Manage Schedules, and add a cron expression.

CI or API

Start the plan from a pipeline or from POST /v1/run.
From the app, open the test plan and select Run Tests. Manual and scheduled runs covers a single test and its dependencies. From a terminal, run qatech run. From another system, call the Start Run API.

​
Pick your CI

Each guide is only the regression setup. The same CI names under PR Testing review a pull request instead.

GitHub

Test Run Action, with blocking.

GitLab

GitLab CI calls the Start Run API.

Bitrise

Mobile build upload, or a web plan.

Envoyer

A deploy hook starts the plan.

API

Bitbucket, Jenkins, CircleCI, or any HTTP client.

​
After the run

Every time a test plan runs, a review agent investigates each test that did not pass, reruns suspected flakes, and writes one verdict with the failures, flaky tests, setup problems and tests to update. Notifications and CI wait for that review, except when you start the plan with /regression in chat. See Regression review agent.

​
How the suite grows

After a GitHub pull request merges, the post-merge agent can promote tests that the review created. Promotion activates the test and labels it regression and auto-added. The agent only changes tests when its prompt asks it to. New GitHub integrations default the agent on. Existing integrations leave it off until you enable Run post-merge agent under Settings → Integrations → GitHub App.

​
While the suite runs

Parallelism, dependency order, and reused browser state: When the tests finish, start with the regression review on the Analysis tab, then How to review test results, Issues, and Notifications.