Conformance Loop
Iteratively improve Fallow analysis accuracy by comparing it with competing tools and...
Install
npx skills add fallow-rs/fallow --skill conformance-loop -g -a claude-code -y- Repository
- fallow-rs/fallow5.3k
- Category
- Engineering
- Updated
- 3d ago
- License
- MIT
- Context cost
- Lightabout 178 tokens
- Runs scripts
- No, instructions only
What it does
Iteratively improve Fallow analysis accuracy by comparing it with competing tools and verified source truth across a stable real-world corpus.
Conformance loop
- Define the capability and stable project corpus.
- Run Fallow and comparison tools with documented equivalent settings.
- Manually verify disagreements against source.
- Classify true positive, false positive, false negative, or model difference.
- Record the classification in
tests/conformance/adjudications.jsonbefore writing the fix. A verdict that lives only in a session or in.plans/is lost, and the daily lane then re-reports the same disagreement as unadjudicated forever. - Implement one general correction with a regression fixture.
- Re-run the full corpus and retain only net improvements.
- Run
review.
Competitor output is a lead, not ground truth.
