Harness Stripping

Every harness component was added to compensate for a specific model failure. Models improve. Components don't retire themselves. The scaffolding that saved you on Sonnet 4.5 may be dead weight, or actively harmful, on Opus 4.6. Strip it deliberately, one piece at a time, and let evals tell you what still earns its keep.

Inspired by Prithvi's March 2026 harness post on evaluator-generator separation and the general "re-test your assumptions each model bump" discipline.

When to apply

• A model upgrade just landed and your harness was tuned for the previous generation.
• A component's justification is "we added this because the model used to do X", and you haven't checked whether it still does X.
• The harness has accreted over months and nobody remembers what half the machinery is for.
• Cost or latency is climbing and you suspect redundant belt-and-suspenders layers.

Procedure

1. Inventory the components. List every distinct piece of scaffolding: prompt sections, tool wrappers, post-hoc validators, retry loops, evaluator personas, structured-output enforcers, sandbox rules. One row per component. Note the failure mode each was added to prevent.

2. Rank by suspicion. Put the components most likely to be obsolete at the top: anything added before the last two model bumps, anything targeting a failure mode you haven't seen recently, anything whose original justification is now folklore.

3. Pick a baseline eval. You need a repeatable metric before you touch anything. Reuse an existing eval set if you have one; otherwise pick 20, 60 tasks representative of production work. Record baseline score, cost, and wall-clock.

4. Strip one component. Only one. Comment it out or gate it behind a flag, don't delete yet. Re-run the eval.