Spec Provenance Review
Flag concrete false-positive proof introduced by changed specs, not test helper or...
Install
npx skills add different-ai/openwork --skill spec-provenance-review -g -a claude-code -y- Repository
- different-ai/openwork24.4k
- Category
- Testing
- Updated
- 3d ago
- License
- See repository
- Context cost
- Lightabout 677 tokens
- Runs scripts
- No, instructions only
What it does
Flag concrete false-positive proof introduced by changed specs, not test helper or channel preferences. Advisory only; never gates Warden clearance.
Review changes under evals/specs/** and evals/worlds/** for one question:
does this diff let a spec pass while the specific behavior it claims to test
is broken?
Test code has a different purpose from production code. Review the validity
of its evidence, not production hardening, abstraction, style, or preferred
helper usage. Channel conventions in evals/README.md are authoring guidance;
a channel mismatch alone is not a finding.
Report a MEDIUM (advisory) finding only when ALL of these hold:
- The changed lines introduce or materially worsen the gap.
- The spec bypasses the behavior under test or asserts evidence unrelated to that behavior.
- You can identify a concrete broken behavior that would still pass, grounded in the spec and relevant implementation. A hypothetical possibility or the availability of a different helper is insufficient.
Examples worth reporting:
- A spec claims a person can submit a form, but directly invokes the API and never submits through the UI, so broken form wiring is not exercised.
- A spec claims a save persists data, but only asserts the seeded value and never observes the result of saving.
- A spec claims visible success, but checks an internal success flag while the implementation demonstrably never renders the result.
Do not report:
