Vaccinate
Qualify a check before it is allowed to clear anything, prove it can detect the...
Install
npx skills add pedrohcgs/claude-code-my-workflow --skill vaccinate -g -a claude-code -y- Repository
- pedrohcgs/claude-code-my-workflow1.7k
- Category
- Testing
- Updated
- 1mo ago
- License
- MIT
- Context cost
- Mediumabout 1,903 tokens
- Runs scripts
- No, instructions only
What it does
Qualify a check before it is allowed to clear anything, prove it can detect the failure it is meant to catch. Seeds known defects into a copy of a real artifact plus a clean control, runs the checker, and reports recall and false-positive rate into a qualification ledger. Use when the user says "does this check work", "qualify the gate", "test my reviewer", "seed defects", "vaccinate", "qualify the checks", "can I trust this review", "does it pass for the right reason", or before relying on any automated check or referee simulation for a decision that matters. NOT a code fixer and NOT a reviewer itself, it grades the grader.
Vaccinate, grade the grader
Twenty bugs were once planted in a working codebase and the review agents were asked to check it again. They reported everything was fine. Recall: 0/20.
A vaccine is a small, controlled dose of error that strengthens the whole system. This skill administers one.
The rule it enforces: an unqualified check is not weak evidence, it is none.
When to run it
- Before a referee simulation, reproducibility gate, or review agent is used to make a decision that matters (a submission, a release, a deposit).
- After changing a checker, a modified gate is unqualified until re-measured.
- On a schedule for gates that guard load-bearing claims. Detection decays as artifacts drift.
Protocol
1. Name the failure
State the defect class the check is supposed to catch. "Catches problems" is not a class. "Detects a coefficient in the text that no longer matches its table" is.
2. Build the seeded set + a clean control
Work on a copy, never the live artifact. Produce:
- N seeded variants, one defect each, drawn from
references/defect-library.md. - At least one clean control, an unmodified copy.
The control is not optional. Without it you measure recall and call it accuracy.
Verify each seed actually violates something. A seed that the artifact already permits creates no defect, and the checker correctly reporting "pass" will look like a broken gate. This is the most common way a qualification run produces a false alarm about itself.
3. Run the checker blind
Run the check or agent against each variant in a fresh context, one variant per run. It
must not know which variant it has, how many defects exist, or that a qualification is
underway. For an AI reviewer, spawn a fresh-context Agent call (not a conversation fork, which would carry the seeded answer key).
