Autoresearch
Autonomous iterative experimentation loop for any programming task.
Install
npx skills add github/awesome-copilot --skill autoresearch -g -a claude-code -y- Repository
- github/awesome-copilot40.7k
- Category
- Testing
- Updated
- 6d ago
- License
- MIT
- Context cost
- Mediumabout 2,563 tokens
- Runs scripts
- No, instructions only
What it does
Autonomous iterative experimentation loop for any programming task. Guides the user through defining goals, measurable metrics, and scope constraints, then runs an autonomous loop of code changes, testing, measuring, and keeping/discarding results. Inspired by Karpathy''s autoresearch. USE FOR: autonomous improvement, iterative optimization, experiment loop, auto research, performance tuning, automated experimentation, hill climbing, try things automatically, optimize code, run experiments, autonomous coding loop. DO NOT USE FOR: one-shot tasks, simple bug fixes, code review, or tasks without a measurable metric.
Autoresearch: Autonomous Iterative Experimentation
An autonomous experimentation loop for any programming task. You define the goal and how to measure it; the agent iterates autonomously -- modifying code, running experiments, measuring results, and keeping or discarding changes -- until interrupted.
This skill is inspired by Karpathy's autoresearch, generalized from ML training to any programming task with a measurable outcome.
Agent Behavior Rules
- DO guide the user through the Setup phase interactively before starting the loop.
- DO establish a baseline measurement before making any changes.
- DO commit every experiment attempt before running it (so it can be reverted cleanly).
- DO keep a results log (TSV) tracking every experiment.
- DO revert changes that do not improve the metric (git reset to last known good).
- DO run autonomously once the loop starts -- never pause to ask "should I continue?".
- DO NOT modify files the user marked as out-of-scope.
- DO NOT skip the measurement step -- every experiment must be measured.
- DO NOT keep changes that regress the metric unless the user explicitly allowed trade-offs.
- DO NOT install new dependencies or make environment changes unless the user approved it.
Phase 1: Setup (Interactive)
Before any experimentation begins, work with the user to establish these parameters. Ask the user directly for each item. Do not assume or skip any.
1.1 Define the Goal
Ask the user:
What are you trying to improve or optimize?
Examples: execution time, memory usage, binary size, test pass rate, code coverage, API response latency, throughput, error rate, benchmark score, build time, bundle size, lines of code, cyclomatic complexity, etc.
Record the user's answer as the goal.
1.2 Define the Metric
Ask the user:
