# Cmux Test Bisect

Find when and why tests broke on main: bisect a Swift package suite on CI across old...

- Repository: https://github.com/manaflow-ai/cmux (28635 stars)
- Category: Testing
- License: see repository
- Last updated: 2026-11-01
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add manaflow-ai/cmux --skill cmux-test-bisect -g -a claude-code -y`
Codex: `npx skills add manaflow-ai/cmux --skill cmux-test-bisect -g -a codex -y`
Cursor: `npx skills add manaflow-ai/cmux --skill cmux-test-bisect -g -a cursor -y`
Gemini CLI: `npx skills add manaflow-ai/cmux --skill cmux-test-bisect -g -a gemini-cli -y`

## Description

Find when and why tests broke on main: bisect a Swift package suite on CI across old commits, read the failure matrix, and decide per test whether the test went stale or the code regressed. Use when a suite is red on main, when PR CI only ran filtered tests and the full suite has drifted, or when asked which PR broke a test.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# cmux test bisect

PR CI runs selected tests, so a package suite can stay red on main for weeks
with nobody noticing. This skill finds the commit behind each failure on CI,
never on the Mac (no full cmux builds on laptops).

App-host (`cmuxTests`) failures on main have their own automatic bisect
(#14510). Use this skill for the SwiftPM package suites under `Packages/iOS` and `Packages/Shared` that `test-ios.yml` can run
(`CMUXMobileCore`, `CmuxSyncStore`, `CmuxMobilePairedMac`, `CmuxMobileChanges`,
`CmuxMobileShell`, `CmuxMobileShellModel`).

## Before you start

1. Get today's failing set from the red run:
   `gh run view <run> --repo manaflow-ai/cmux --log-failed`, then grep `✘ Test .* failed after`.
   Run the full suite twice if you have not already. A test that appears in
   only one run is a flake candidate, not a regression.
2. Search for an existing fix: `gh search prs --repo manaflow-ai/cmux --state open '<test name>'`.
3. Work in your own worktree from `upstream/main`. If `git log` stops at a
   recent commit, the checkout is shallow: `git fetch --shallow-since=<date> upstream main`.

## Run the bisect

```bash
T=scripts/ci/package_bisect.py
python3 $T --package CmuxMobileShell start --points 6 <old-sha>..upstream/main
python3 $T --package CmuxMobileShell status --wait     # up to 45 min
python3 $T --package CmuxMobileShell next --dispatch   # split each break window
python3 $T --package CmuxMobileShell cleanup           # delete probe branches
```

The rest: https://github.com/manaflow-ai/cmux/blob/HEAD/skills/cmux-test-bisect/SKILL.md