Audit Reproducibility

Compare numeric claims in a manuscript (point estimates, standard errors, p-values, counts) against the actual outputs produced by the analysis pipeline. Report PASS / FAIL per claim against the tolerance thresholds defined in .claude/rules/replication-protocol.md.

Core principle: If the paper says ATT = -1.632 (0.584) and the code produces -1.628 (0.591), we verify, numerically, that the difference is within the documented tolerance. No more "looks close enough" eyeballing.

Two directions, not one. Vertically, each claim is checked against the output that produced it. Horizontally, it is checked against every other artifact that displays the same number, the supplement table, the slide deck, the poster. The vertical check passes contentedly while a deck quotes last month's value; only the horizontal one catches that. Declared displays live in the passport's appears_in list, see replication-protocol.md → The horizontal check.

When to use

• Before submission. Catches the "I updated the analysis but forgot to update Table 2" bug.
• Before presenting or teaching from the same numbers. Catches the deck, the poster, or the supplement that was never regenerated after the last rerun.
• Before releasing a replication package. Verifies the code actually reproduces the paper.
• After a major revision. Ensures the paper still matches the latest code.
• Quality-gate in /commit. Pair with a pre-commit invocation on manuscript + analysis changes.

Inputs