Compare Test Runs

Quick Start

You'll typically receive two test run identifiers. Follow these steps:

1. Run tuist test show <id> --json for both base and head test runs.
2. Run tuist test module list <test-run-id> --json and tuist test suite list <test-run-id> --json to get module and suite breakdowns.
3. Run tuist test case run list <identifier> --json to get individual test case results.
4. Compare failures, flaky tests, durations, and overall status.
5. Inspect failing test cases with tuist test case run show <id> --json.
6. Summarize findings with actionable recommendations.

Step 1: Resolve Test Runs

If base/head are test run IDs or dashboard URLs

Fetch each directly:

tuist test show <base-id> --json
tuist test show <head-id> --json

If base/head are branch names

List recent test runs on each branch to identify test run IDs:

tuist test list --git-branch <base-branch> --json --page-size 5
tuist test list --git-branch <head-branch> --json --page-size 5

Pick the latest test run ID from each branch's results.

Defaults

• If no base is provided, use the project's default branch (usually main).
• If no head is provided, detect the current git branch.

Step 2: Compare Top-Level Metrics

After fetching both test runs, compare:

Metric	What to check
status	Flag if base passed but head failed
duration	Flag if head is >10% slower
total_test_count	Note if test count changed (new or removed tests)
failed_test_count	Compare failure counts
flaky_test_count	Compare flaky counts
avg_test_duration	Flag significant changes

Step 3: Get Module and Suite Breakdowns

Fetch module and suite-level results for both test runs to understand which areas regressed:

tuist test module list <base-test-run-id> --json
tuist test module list <head-test-run-id> --json