Skills

Deflake

Stabilize a flaky test with a minimal, assertion-preserving fix, never by weakening or...

Install

npx skills add QwenLM/qwen-code --skill deflake -g -a claude-code -y
Repository
QwenLM/qwen-code28.9k
Category
Testing
Updated
3d ago
License
Apache-2.0
Context cost
Lightabout 819 tokens
Runs scripts
No, instructions only

What it does

Stabilize a flaky test with a minimal, assertion-preserving fix, never by weakening or deleting the check.

Deflake a flaky test

A deflake: issue names ONE test that has been observed failing and then passing on a rerun of the same commit, the definitive flaky signature. Your job is to make that test deterministic without changing what it verifies.

The issue body carries the test identity (file + name) and the observed failure signature (e.g. Test timed out in 5000ms, Timed out waiting for …, an order-dependent assertion, a wall-clock/random-dependent value). Read the test, reproduce the mechanism in your head, and apply the SMALLEST fix from the allowed set below that removes the nondeterminism.

The only allowed fixes

  1. Raise a timeout / poll budget. A test that blows vitest's default under CI contention (fully-mocked or I/O-bound, not a perf test) gets a generous per-test testTimeout (3rd arg to it), or its internal poll loop is given a real wall-clock budget instead of a fixed iteration count (a fixed count of setImmediate turns elapses in milliseconds and races real I/O).
  2. Stabilize timing / waiting. Replace a bare setTimeout/fixed sleep with an explicit await of the real condition (vi.waitFor, a resolved promise, an event). Pre-warm a lazy load (e.g. a WASM runtime) in beforeAll so per-test time doesn't include first-load cost.
  3. Make randomness / time deterministic. Seed the RNG, vi.useFakeTimers() / mock Date.now, or pin the input so a value that depends on the real clock or Math.random can't drift.
  4. Isolate / serialize interference. Give tests that collide on a shared resource (a same-named tempdir, a fixed port, a global singleton) unique per-test resources, or serialize them.

Hard rules

Read the whole SKILL.md on GitHub

More in Testing