AAAI Experiments

Use this before submission to ensure empirical evidence supports the AI contribution. AAAI reviewers may come from adjacent AI subfields, so experiments must be interpretable beyond one benchmark community.

Experiment audit

• Map every experimental block to a claim in the introduction.
• Compare against strong, recent, and fairly tuned baselines.
• Include ablations that isolate mechanisms rather than removing multiple components at once.
• Report uncertainty, variance, and statistical tests when small differences matter.
• Test robustness to data split, prompt, seed, environment, user population, or distribution shift when relevant.
• For human evaluation, document task, instructions, annotator pool, quality control, aggregation, and ethics/IRB status.
• Report compute, hardware, data access, model size, and training/inference cost.

Claim-to-evidence ledger

Build this table before adding new experiments. It keeps the AAAI evidence package aligned with the main text and with the reproducibility checklist.

Manuscript claim	Required evidence	Phase-1 risk if missing	Checklist hook
New AI capability	benchmark + qualitative failure cases	broad reviewer sees only engineering	datasets, metrics, baselines
Better mechanism	single-factor ablations	gain looks like tuning luck	ablation and hyperparameter answers
Robust deployment	shift / seed / subgroup stress test	result seems brittle	variance, compute, environment
Social-impact or safety claim	stakeholder, harm, and misuse analysis	ethical claim looks asserted	ethics, limitations, data access

For each row, mark ready / weak / missing and name the fastest fix that can be run before the supplementary-material deadline. Do not leave a claim in the abstract if its evidence row is weak.

AAAI-specific review pressure