Stat Result Validator

Overview

Use this skill after formulation, method proposal, theory, experimental evaluation, comparison, and result synthesis. It checks whether the final result is supported by a coherent statistical research chain.

Artifact Checks

Required for all topics:

progress/<TOPIC_ID>/step0_problem_formulation.md
progress/<TOPIC_ID>/step1_method_proposal.md
progress/<TOPIC_ID>/step2_theory_analysis.md
progress/<TOPIC_ID>/step3_experimental_evaluation.md
progress/<TOPIC_ID>/step4_comparison.md
progress/<TOPIC_ID>/step5_result_synthesis.md
progress/<TOPIC_ID>/step6_quality_audit.md
experiments/<TOPIC_ID>/config.yaml
experiments/<TOPIC_ID>/results/metrics.json
experiments/<TOPIC_ID>/results/run_manifest.json
experiments/<TOPIC_ID>/results/comparison_summary.md
experiments/<TOPIC_ID>/results/claim_verdicts.json
experiments/<TOPIC_ID>/report/paper.md
experiments/<TOPIC_ID>/README.md

Analysis-specific source files and raw outputs are determined by the experiment plan and should live under experiments/<TOPIC_ID>/src/ and experiments/<TOPIC_ID>/results/.

Formulation Checks

The formulation must define:

• Observed data and sampling regime
• Data model or data source
• Target parameter, decision, prediction, or risk
• Assumptions
• Claims or hypotheses
• Evaluation criteria
• Theory targets

Blocking failures:

• No target or estimand.
• Claims cannot be measured.
• Assumptions are absent or incompatible with the proposed method.
• Evaluation criteria do not answer the research question.

Method Checks

Verify that:

• The proposed method addresses the formulated target.
• Baselines are meaningful.
• Ablations isolate important design choices.
• Diagnostics are specified for likely failure modes.
• Method outputs match the metrics and theory targets.

Theory Checks

Theory may be rigorous or partial, but it must be explicit.

Check for: