Fix Flaky Tests

Quick Start

You'll typically receive a Tuist test case URL or identifier. Follow these steps to investigate and fix it:

1. Run tuist test case show <id-or-identifier> --json to get reliability metrics for the test.
2. Run tuist test case run list Module/Suite/TestCase --flaky --json to see flaky run patterns.
3. Run tuist test case run show <test-case-run-id> --json on failing flaky runs to get failure messages and file paths.
4. Read the test source at the reported path and line, identify the flaky pattern, and fix it.
5. Verify by running the test multiple times to confirm it passes consistently.

If no specific test is provided, start with the Discovery section below.

Discovery

When no specific test case is provided, find all flaky tests in the project:

tuist test case list --flaky --json --page-size 50

This returns all test cases currently flagged as flaky. Key fields:

• module.name / suite.name / name, the test identifier
• avg_duration, helps prioritize (fix fast unit tests first)
• is_quarantined, whether the test is already quarantined

Triage strategy:

1. Group tests by suite, multiple flaky tests in the same suite often share a root cause.
2. Check if failures share a test_run_id, tests that all failed in the same run may have been killed by a process crash, not individual test bugs.
3. Look at failure messages to categorize: test logic bugs vs infrastructure issues (network errors, server 502s, conflicts on retry).

Investigation

1. Get test case metrics

You can pass either the UUID or the Module/Suite/TestCase identifier:

tuist test case show <id> --json
tuist test case show Module/Suite/TestCase --json

Key fields:

• reliability_rate, percentage of successful runs (higher is better)
• flakiness_rate, percentage of runs marked flaky in the last 30 days
• total_runs / failed_runs, volume context
• last_status, current state