Test plans

​

Overview

• Organizing related test cases into logical groups
• Enabling automated execution on schedules
• Supporting API-driven test automation
• Managing test dependencies efficiently

​

Creating a Test Plan

1. Navigate to the Test Plans section
2. Click the “Create Test Plan” button in the top right
3. Provide the following information:
   • Name: A clear, descriptive name for your test plan
   • Description (optional): Additional context about the test plan’s purpose
   • Owner (optional): Assign a team member responsible for this test plan. See Ownership
4. Click “Save” to create your test plan

​

Find the test plan short ID

1. Go to Test Plans
2. Click on a test plan
3. Check the URL: https://app.qa.tech/.../test-plans/abc123
4. Use the test plan short ID in API payloads (for example: pln_abc123)

​

Adding Test Cases to a Test Plan

• Dependencies will be automatically included in runs, even if not visible in the UI
• Consider creating separate plans for:
  • Frequently run core tests
  • Less frequent tests that run on a schedule

​

Execution Methods

• From CI: pick your CI under Regression Testing.
• From the API: call the Start Run API.
• From the app or on a schedule: see Running Tests.

curl -X POST https://api.qa.tech/v1/run \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "testPlanShortId": "pln_abc123"
  }'

​

Performance & Parallel Execution

• Automatic parallelization: Tests run concurrently by default when executing a test plan, significantly reducing the time required to complete a full test suite
• Dependency-aware: Parallel execution respects the dependency graph, ensuring dependent tests run in the correct order while maximizing parallelization opportunities
• Configurable limits: Set Maximum Concurrent Tests on each environment to cap parallel runs against staging, production, or other URLs
• Auto-scaling: When no limit is set, the system scales up to approximately 100 concurrent agents based on demand
• Intelligent scheduling: Independent tests execute simultaneously while dependent tests wait for their prerequisites to complete

​

Best Practices

1. Optimize Test Organization:
   • Group related test cases together
   • Create separate plans for different testing frequencies
   • Keep plans focused and maintainable
2. Dependency Management:
   • Be aware that dependent test cases will be included automatically
   • Review dependencies when setting up new test plans
3. Schedule and Workflow Optimization:
   • Schedule less frequent tests during off-peak hours
   • Configure GitHub workflows to run tests at key points in your development process (deployments, PRs, merges)
4. Managing Different Versions and Releases:
   • Use Test Scenarios to define the specific test cases to run for a given version or release
   • The same Test Plan can be reused across different environments by overriding environment URLs via API (see Start Run API) or updating test plan parameters in the UI
   • Note: Both environment and device preset overrides are supported via API. See Start Run API for details on using applications overrides.
5. Notification Configuration:
   • Configure email, Slack, and Microsoft Teams notifications at the test plan level
   • Override the project’s default Slack channel for test plan-specific routing
   • Add organization members as email recipients to ensure stakeholders receive run completion notifications
   • See Notifications for detailed setup instructions

​

Test Plan Parameters

​

Configuration Levels

Project Settings (defaults for all runs)
└── Test Plan Parameters (override per test plan)
    └── Per-Run API (override per individual run)

​

Configuring Parameters in the UI

​

How Tests Execute with Multiple Applications

• Test Plan contains: Test A (App 1), Test B (App 1), Test C (App 2)
• Parameters: App 1 → Production, App 2 → Staging
• Execution:
  • Test A runs on Production (App 1’s environment)
  • Test B runs on Production (App 1’s environment)
  • Test C runs on Staging (App 2’s environment)

​

API Overrides

{
  "testPlanShortId": "pln_abc123",
  "applications": [
    {
      "applicationShortId": "app_gXeBl2",
      "environment": { "url": "https://preview.example.com" },
      "devicePresetShortId": "preset_abc123"
    }
  ]
}