
Deflake
Stabilize a flaky test with a minimal, assertion-preserving fix, never by weakening or...
Install
npx skills add QwenLM/qwen-code --skill deflake -g -a claude-code -y- Repository
- QwenLM/qwen-code28.9k
- Category
- Testing
- Updated
- 3d ago
- License
- Apache-2.0
- Context cost
- Lightabout 819 tokens
- Runs scripts
- No, instructions only
What it does
Stabilize a flaky test with a minimal, assertion-preserving fix, never by weakening or deleting the check.
Deflake a flaky test
A deflake: issue names ONE test that has been observed failing and then
passing on a rerun of the same commit, the definitive flaky signature. Your
job is to make that test deterministic without changing what it verifies.
The issue body carries the test identity (file + name) and the observed failure
signature (e.g. Test timed out in 5000ms, Timed out waiting for …, an
order-dependent assertion, a wall-clock/random-dependent value). Read the test,
reproduce the mechanism in your head, and apply the SMALLEST fix from the
allowed set below that removes the nondeterminism.
The only allowed fixes
- Raise a timeout / poll budget. A test that blows vitest's default under
CI contention (fully-mocked or I/O-bound, not a perf test) gets a generous
per-test
testTimeout(3rd arg toit), or its internal poll loop is given a real wall-clock budget instead of a fixed iteration count (a fixed count ofsetImmediateturns elapses in milliseconds and races real I/O). - Stabilize timing / waiting. Replace a bare
setTimeout/fixedsleepwith an explicitawaitof the real condition (vi.waitFor, a resolved promise, an event). Pre-warm a lazy load (e.g. a WASM runtime) inbeforeAllso per-test time doesn't include first-load cost. - Make randomness / time deterministic. Seed the RNG,
vi.useFakeTimers()/ mockDate.now, or pin the input so a value that depends on the real clock orMath.randomcan't drift. - Isolate / serialize interference. Give tests that collide on a shared resource (a same-named tempdir, a fixed port, a global singleton) unique per-test resources, or serialize them.


