Cmux Test Bisect
Find when and why tests broke on main: bisect a Swift package suite on CI across old...
Install
npx skills add manaflow-ai/cmux --skill cmux-test-bisect -g -a claude-code -y- Repository
- manaflow-ai/cmux28.6k
- Category
- Testing
- Updated
- 3d ago
- License
- See repository
- Context cost
- Lightabout 1,392 tokens
- Runs scripts
- No, instructions only
What it does
Find when and why tests broke on main: bisect a Swift package suite on CI across old commits, read the failure matrix, and decide per test whether the test went stale or the code regressed. Use when a suite is red on main, when PR CI only ran filtered tests and the full suite has drifted, or when asked which PR broke a test.
cmux test bisect
PR CI runs selected tests, so a package suite can stay red on main for weeks with nobody noticing. This skill finds the commit behind each failure on CI, never on the Mac (no full cmux builds on laptops).
App-host (cmuxTests) failures on main have their own automatic bisect
(#14510). Use this skill for the SwiftPM package suites under Packages/iOS and Packages/Shared that test-ios.yml can run
(CMUXMobileCore, CmuxSyncStore, CmuxMobilePairedMac, CmuxMobileChanges,
CmuxMobileShell, CmuxMobileShellModel).
Before you start
- Get today's failing set from the red run:
gh run view <run> --repo manaflow-ai/cmux --log-failed, then grep✘ Test .* failed after. Run the full suite twice if you have not already. A test that appears in only one run is a flake candidate, not a regression. - Search for an existing fix:
gh search prs --repo manaflow-ai/cmux --state open '<test name>'. - Work in your own worktree from
upstream/main. Ifgit logstops at a recent commit, the checkout is shallow:git fetch --shallow-since=<date> upstream main.
Run the bisect
T=scripts/ci/package_bisect.py
python3 $T --package CmuxMobileShell start --points 6 <old-sha>..upstream/main
python3 $T --package CmuxMobileShell status --wait # up to 45 min
python3 $T --package CmuxMobileShell next --dispatch # split each break window
python3 $T --package CmuxMobileShell cleanup # delete probe branches
