Understanding Notifications
Notification Types
Delivery Channels
Configuration Levels
- Project Level - Set default Slack channel and Teams webhook. All runs use these unless overridden.
- Test Plan Level - Override Slack channel, add email recipients, configure Teams webhook per test plan.
- Per-Run API - Override Slack channels dynamically when triggering runs via API. Useful for CI/CD.
Per-run API overrides only support Slack. Email and Teams notifications cannot
be overridden via API - configure them at the test plan level instead.
What You Receive
Run Completion Notifications
All run completion notifications (Slack, Email, Teams) include:
You receive a notification for every completed run - whether triggered manually, through CI/CD, or via schedule.
Weekly Summary Reports
Weekly summaries are separate from run completion notifications. They provide organization-wide metrics to all members. When: Every Monday at 6 AM Central European TimeWho: All organization members (unsubscribe via email footer) Contents:
- Tests executed count
- Pass percentage
- Time saved estimate
- Suggested new tests to add
- Summary of test runs
Common Patterns
Team-Wide Slack Alerts
Goal: Everyone on the project sees test results in a shared channel. Setup: Configure default Slack channel at Project Settings → Integrations → Slack. All runs notify this channel automatically.Stakeholder Email Notifications
Goal: Product managers or executives receive email updates for specific test plans. Setup: Open test plan → Notification settings → Add email recipients. Choose “Only on failure” to reduce email volume.CI/CD-Triggered Notifications
Goal: Tests triggered via GitHub Actions or GitLab CI notify the team automatically. Setup: Runs triggered via API use project defaults. No additional configuration needed - notifications flow to configured channels. Guides: GitHub Actions | GitLab CI | Start Run APIPR-Specific Slack Channels
Goal: Route notifications for a specific PR to a dedicated Slack channel. Setup: Use per-run API overrides to specify channels dynamically:Failure-Only Routing
Goal: Reduce noise by only notifying on failures. Setup:- Email/Teams: Set “Only on failure” in test plan notification settings
- Slack API override: Use
"notifyOn": "failure"in the notifications array
Decision Guide
Use this flowchart to determine where to configure notifications:Setup by Channel
Slack
1
Connect Slack to Organization
Go to Organization Settings → Connections and connect your Slack workspace.
2
Set Project Default Channel
Go to Project Settings → Integrations → Slack and select a default channel.
3
(Optional) Override Per Test Plan
Open a test plan → Click settings icon (gear) next to Notifications → Select
a different Slack channel.
1
Open Test Plan
Navigate to your test plan from the Test Plans page.
2
Open Notification Settings
Click the settings icon (gear) next to Notifications on the test plan
page.
3
Add Recipients
Search for organization members by name or email and add them as recipients.
4
Choose Frequency
Select Every completion (all runs) or Only on failure (failures
only).
Email recipients must be organization members. External email addresses are
not supported.
Microsoft Teams
Teams notifications use incoming webhooks. See Microsoft Teams Integration for detailed webhook setup instructions.1
Create Incoming Webhook in Teams
Follow Microsoft’s guide to create an incoming webhook for your Teams
channel.
2
Add Webhook URL
Go to Project Settings → Integrations → Microsoft Teams, or configure at the
test plan level via Notification Settings.
3
(Optional) Configure Frequency
Choose Every completion or Only on failure in test plan settings.
Advanced Configuration
Per-Test-Plan Settings
Access test plan notification settings by clicking the settings icon (gear) next to Notifications on any test plan page. You can configure:
Each test plan can have different notification settings, allowing you to:
- Route critical test plans to dedicated alert channels
- Add stakeholders to specific test plans
- Use different Teams channels for different test plans
Per-Run API Overrides (Slack Only)
When triggering runs via the Start Run API, you can override Slack notification destinations for that specific run.
For complete API documentation, see Start Run API Reference.