Improve Skill Quality
Diagnoses and fixes skills in the dotnet/skills repository that lose to their own...
Install
npx skills add dotnet/skills --skill improve-skill-quality -g -a claude-code -y- Repository
- dotnet/skills5.8k
- Category
- Testing
- Updated
- 3d ago
- License
- MIT
- Context cost
- Mediumabout 3,000 tokens
- Runs scripts
- No, instructions only
What it does
Diagnoses and fixes skills in the dotnet/skills repository that lose to their own baseline, fail to activate, time out, or return "no credible improvement". Use when an evaluation verdict is a regression or underpowered, when a skill regressed after a change, when /evaluate reports no results, or when deciding whether a weak skill should be strengthened or retired. Do not use for scaffolding a brand-new skill (use create-skill) or a brand-new eval (use create-skill-test).
Improve Skill Quality
Turn a failing or unconvincing evaluation into a targeted fix. The single most common mistake in this repo is rewriting skill prose in response to a verdict whose real cause was the eval, the fixtures, or the harness. Classify first, then fix.
When to Use
- An evaluation verdict is a regression, underpowered, or "no credible improvement".
- A skill wins in the isolated arm but not in the plugin arm, or is reported "not activated".
/evaluatereports "Evaluation ran but produced no results".- A skill scores well but costs too much (tokens, turns, wall time, plugin menu budget).
- Deciding whether to strengthen or retire a persistently weak skill.
When Not to Use
- Creating a new skill from scratch, use
create-skill. - Creating a new
eval.yamlfrom scratch, usecreate-skill-test. - Changing the harness itself (
eng/skill-validator,eng/vally-adapter,evaluation*.yml).
Inputs
| Input | Required | Description |
|---|---|---|
| Verdict evidence | Yes | The /evaluate PR comment, or results.json from the run artifacts |
| Losing trial transcripts | Yes for content fixes | Baseline vs. skilled output plus the judge's stated reason |
| Stimulus-vote W/T/L and repeated-run W/T/L | Yes | Separates cross-task evidence from reliability |
| Activation status per arm | Yes | Isolated and plugin activation are different failures |
Workflow
Step 1: Get the evidence before forming a hypothesis
Read InvestigatingResults.md for how to
download artifacts and read results.json. Extract, per failing stimulus:
- authoritative stimulus-vote W/T/L and separate repeated-run W/T/L
- activation status in the isolated and plugin arms, separately
- the judge's verbatim reason on each losing trial
- whether any trial errored, timed out, or produced empty output
