Aeon Skill Evals
Validate the output of any installed skill against an assertion manifest, word counts...
Install
npx skills add BankrBot/skills --skill aeon-skill-evals -g -a claude-code -y- Repository
- BankrBot/skills1.2k
- Category
- Engineering
- Updated
- 5d ago
- License
- See repository
- Context cost
- Lightabout 514 tokens
- Runs scripts
- No, instructions only
What it does
Validate the output of any installed skill against an assertion manifest, word counts, required patterns, forbidden phrases, required sections, source citation. Detects regressions by diffing vs prior runs (NEW_FAIL / NEW_PASS / CHRONIC / STABLE_FAIL). Bootstrap mode generates a starter manifest from a skill's recent successful runs so manifests aren't written speculatively. Triggers: "evaluate this skill's output", "check skill X for regressions", "bootstrap evals for Y", "did this skill output pass quality gates".
aeon-skill-evals
Quality net for installed skills. Each skill can declare an assertion manifest; outputs are checked against it; failing assertions surface regressions and route concrete fixes.
Manifest format
token-movers:
min_words: 200
required_patterns: ["Top movers", "24h"]
forbidden_patterns: ["I cannot", "as an AI"]
must_cite_source: true
min_distinct_items: 5
narrative-tracker:
min_words: 400
required_sections: ["TRANSITIONS", "POSITIONS", "MAP"]
forbidden_patterns: ["exciting", "consider"]
must_have_position_call: true
Supported assertions: min_words / max_words, required_patterns / forbidden_patterns, required_sections, must_cite_source, min_distinct_items, output_pattern (regex), and per-skill-family custom binary checks.
Operations
eval, run every manifest-defined skill against its latest output.eval --skill=NAME, one skill.bootstrap --skill=NAME, generate a starter manifest from recent successful runs.
Regression states
| State | Action |
|---|---|
NEW_FAIL | Passing last run, failing now. Severity scales with pass streak. |
NEW_PASS | Failing last run, passing now. Log the win. |
CHRONIC | Failing > 3 consecutive runs. Recommend operator review. |
STABLE_FAIL | Always failing. Manifest assertion mismatch, flag for review. |
State in local evals-state.json.
Bootstrap mode
Samples last 5 successful runs of a skill. Computes:
min_wordsat p25 of historical runs.- Required patterns from common section headers.
- Forbidden patterns from default list (refusals, hedging filler).
Emits the proposed manifest for review. Never auto-commits, assertions need a human signoff.
