# Compare Test Runs

Compares two test runs to identify new failures, newly flaky tests, fixed tests, and...

- Repository: https://github.com/tuist/tuist (5859 stars)
- Category: Testing
- License: see repository
- Last updated: 2026-11-01
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add tuist/tuist --skill compare-test-runs -g -a claude-code -y`
Codex: `npx skills add tuist/tuist --skill compare-test-runs -g -a codex -y`
Cursor: `npx skills add tuist/tuist --skill compare-test-runs -g -a cursor -y`
Gemini CLI: `npx skills add tuist/tuist --skill compare-test-runs -g -a gemini-cli -y`

## Description

Compares two test runs to identify new failures, newly flaky tests, fixed tests, and duration regressions. Can be invoked with test run IDs, dashboard URLs, or branch names.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Compare Test Runs

## Quick Start

You'll typically receive two test run identifiers. Follow these steps:

1. Run `tuist test show <id> --json` for both base and head test runs.
2. Run `tuist test module list <test-run-id> --json` and `tuist test suite list <test-run-id> --json` to get module and suite breakdowns.
3. Run `tuist test case run list <identifier> --json` to get individual test case results.
4. Compare failures, flaky tests, durations, and overall status.
5. Inspect failing test cases with `tuist test case run show <id> --json`.
6. Summarize findings with actionable recommendations.

## Step 1: Resolve Test Runs

### If base/head are test run IDs or dashboard URLs

Fetch each directly:

```bash
tuist test show <base-id> --json
tuist test show <head-id> --json
```

### If base/head are branch names

List recent test runs on each branch to identify test run IDs:

```bash
tuist test list --git-branch <base-branch> --json --page-size 5
tuist test list --git-branch <head-branch> --json --page-size 5
```

Pick the latest test run ID from each branch's results.

### Defaults

- If no base is provided, use the project's default branch (usually `main`).
- If no head is provided, detect the current git branch.

## Step 2: Compare Top-Level Metrics

After fetching both test runs, compare:

| Metric | What to check |
|---|---|
| `status` | Flag if base passed but head failed |
| `duration` | Flag if head is >10% slower |
| `total_test_count` | Note if test count changed (new or removed tests) |
| `failed_test_count` | Compare failure counts |
| `flaky_test_count` | Compare flaky counts |
| `avg_test_duration` | Flag significant changes |

## Step 3: Get Module and Suite Breakdowns

Fetch module and suite-level results for both test runs to understand which areas regressed:

```bash
tuist test module list <base-test-run-id> --json
tuist test module list <head-test-run-id> --json

The rest: https://github.com/tuist/tuist/blob/HEAD/skills/skills/compare-test-runs/SKILL.md