
Evaluate Optimization Candidates
Runs a paired baseline evaluation of an optimization candidate after approval.
Install
npx skills add JuliusBrussee/caveman --skill caveman-optimize -g -a claude-code -y- Repository
- JuliusBrussee/caveman117k
- Category
- Data and ML
- Updated
- 4d ago
- License
- Apache-2.0
- Context cost
- Lightabout 1,116 tokens
- Runs scripts
- No, instructions only
What it does
Turn a Caveman optimization observation into an operator-chosen candidate with a paired baseline evaluation. Use when asked to inspect or evaluate a Caveman optimization report. Needs explicit approval.
Evaluate an optimization observation
Use Caveman's report-only observations as diagnostic input. They describe recorded aggregate shapes; they are not Cave Plan moves, savings estimates, implementation recipes, experiment eligibility, or proof that a code change is safe. Keep the workflow operator-chosen and evidence-first.
1. Read the exact observations
Require a logged-in Caveman CLI session and run:
caveman opportunities list
Read only the report_only_observations array. Do not select from the lifecycle
data array. Preserve each server-provided title and observation verbatim.
Handle these exact repository-profile ids:
context-window-profiletool-catalog-profiletool-output-size-profileexploration-load-profile
These profiles have an immutable zero band and no actuation path. Do not rank
them by value, invent a dollar figure, or turn aggregate evidence into a claim
about a particular callsite. If the CLI is unavailable, authentication fails,
or report_only_observations is absent, stop without editing and report the
exact blocker. Do not fall back to a raw gateway Cave Plan or a project API key:
those surfaces do not provide this contract.
Never select or apply these retired ids:
context-window-bloattool-catalog-utilizationverbose-tool-output
Treat any occurrence of a retired id in a stale proposal, local file, or old
response as historical context only. Never revive its money, recipe, or
lifecycle claim. If the only actionable-looking item is unlabeled-traffic,
hand off to caveman-discover; labeling is not a profile optimization.
2. Ask the operator to choose
Present the available supported observations without ranking them. Include the
id, the exact title, the exact observation, and last_seen_at. Ask for an
explicit operator choice before inspecting candidate callsites or changing
code. If no supported current observation exists, stop with no edit.
