# Aaai Experiments

Use when designing or auditing AAAI experiments for the broad-AI program committee...

- Repository: https://github.com/brycewang-stanford/Awesome-Journal-Skills (1346 stars)
- Category: Design
- License: MIT
- Last updated: 2026-09-27
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add brycewang-stanford/Awesome-Journal-Skills --skill aaai-experiments -g -a claude-code -y`
Codex: `npx skills add brycewang-stanford/Awesome-Journal-Skills --skill aaai-experiments -g -a codex -y`
Cursor: `npx skills add brycewang-stanford/Awesome-Journal-Skills --skill aaai-experiments -g -a cursor -y`
Gemini CLI: `npx skills add brycewang-stanford/Awesome-Journal-Skills --skill aaai-experiments -g -a gemini-cli -y`
Claude Code plugin marketplace: `/plugin marketplace add brycewang-stanford/Awesome-Journal-Skills`

## Description

Use when designing or auditing AAAI experiments for the broad-AI program committee, including baselines, ablations, statistical significance, robustness, human evaluation, AI-for-Social-Impact and alignment/safety evidence, compute and cost reporting, and reproducibility-checklist alignment for Phase-1 survival.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# AAAI Experiments

Use this before submission to ensure empirical evidence supports the AI contribution. AAAI
reviewers may come from adjacent AI subfields, so experiments must be interpretable beyond one
benchmark community.

## Experiment audit

- Map every experimental block to a claim in the introduction.
- Compare against strong, recent, and fairly tuned baselines.
- Include ablations that isolate mechanisms rather than removing multiple components at once.
- Report uncertainty, variance, and statistical tests when small differences matter.
- Test robustness to data split, prompt, seed, environment, user population, or distribution shift
  when relevant.
- For human evaluation, document task, instructions, annotator pool, quality control, aggregation,
  and ethics/IRB status.
- Report compute, hardware, data access, model size, and training/inference cost.

## Claim-to-evidence ledger

Build this table before adding new experiments. It keeps the AAAI evidence package aligned with
the main text and with the reproducibility checklist.

| Manuscript claim | Required evidence | Phase-1 risk if missing | Checklist hook |
| --- | --- | --- | --- |
| New AI capability | benchmark + qualitative failure cases | broad reviewer sees only engineering | datasets, metrics, baselines |
| Better mechanism | single-factor ablations | gain looks like tuning luck | ablation and hyperparameter answers |
| Robust deployment | shift / seed / subgroup stress test | result seems brittle | variance, compute, environment |
| Social-impact or safety claim | stakeholder, harm, and misuse analysis | ethical claim looks asserted | ethics, limitations, data access |

For each row, mark **ready / weak / missing** and name the fastest fix that can be run before the
supplementary-material deadline. Do not leave a claim in the abstract if its evidence row is weak.

## AAAI-specific review pressure

The rest: https://github.com/brycewang-stanford/Awesome-Journal-Skills/blob/HEAD/AAAI-Skills/skills/aaai-experiments/SKILL.md