# Ablation Study Planner

Plans ablation studies once main results support a claim, for paper submission.

- Repository: https://github.com/wanshuiyin/Auto-claude-code-research-in-sleep (18443 stars)
- Category: Data and ML
- License: MIT
- Last updated: 2026-11-01
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add wanshuiyin/Auto-claude-code-research-in-sleep --skill ablation-planner -g -a claude-code -y`
Codex: `npx skills add wanshuiyin/Auto-claude-code-research-in-sleep --skill ablation-planner -g -a codex -y`
Cursor: `npx skills add wanshuiyin/Auto-claude-code-research-in-sleep --skill ablation-planner -g -a cursor -y`
Gemini CLI: `npx skills add wanshuiyin/Auto-claude-code-research-in-sleep --skill ablation-planner -g -a gemini-cli -y`
Claude Code plugin marketplace: `/plugin marketplace add wanshuiyin/Auto-claude-code-research-in-sleep`

## Description

Use when main results pass result-to-claim (claim_supported=yes or partial) and ablation studies are needed for paper submission.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Ablation Planner

Systematically design ablation studies that answer the questions reviewers will ask. Codex leads the design (reviewer perspective), CC reviews feasibility and implements.

## Context: $ARGUMENTS

## When to Use

- Main results pass `/result-to-claim` with claim_supported = yes or partial
- User explicitly requests ablation planning
- `/auto-review-loop` reviewer identifies missing ablations

## Workflow

### Step 1: Prepare Context

CC reads available project files to build the full picture:
- Method description and components (from `idea-stage/docs/research_contract.md`, legacy `docs/research_contract.md`, or project CLAUDE.md)
- Current experiment results (from EXPERIMENT_LOG.md, EXPERIMENT_TRACKER.md, or W&B)
- Confirmed and intended claims (from result-to-claim output or project notes)
- Available compute resources (from CLAUDE.md server config, if present)

### Step 2: Codex Designs Ablations

```
mcp__codex__codex:
  model: gpt-6-astra
  config: {"model_reasoning_effort": "xhigh"}
  prompt: |
    You are a rigorous ML reviewer planning ablation studies.
    Given this method and results, design ablations that:

    1. Isolate the contribution of each novel component
    2. Answer questions reviewers will definitely ask
    3. Test sensitivity to key hyperparameters
    4. Compare against natural alternative design choices

    Method: [description from project files]
    Components: [list of removable/replaceable components]
    Current results: [key metrics from experiments]
    Claims: [what we claim and current evidence]

    For each ablation, specify:
    - name: what to change (e.g., "remove module X", "replace Y with Z")
    - what_it_tests: the specific question this answers
    - expected_if_component_matters: what we predict if the component is important
    - priority: 1 (must-run) to 5 (nice-to-have)

The rest: https://github.com/wanshuiyin/Auto-claude-code-research-in-sleep/blob/HEAD/skills/ablation-planner/SKILL.md