Skills

Evaluate Optimization Candidates

Runs a paired baseline evaluation of an optimization candidate after approval.

Install

npx skills add JuliusBrussee/caveman --skill caveman-optimize -g -a claude-code -y
Repository
JuliusBrussee/caveman117k
Category
Data and ML
Updated
4d ago
License
Apache-2.0
Context cost
Lightabout 1,116 tokens
Runs scripts
No, instructions only

What it does

Turn a Caveman optimization observation into an operator-chosen candidate with a paired baseline evaluation. Use when asked to inspect or evaluate a Caveman optimization report. Needs explicit approval.

Evaluate an optimization observation

Use Caveman's report-only observations as diagnostic input. They describe recorded aggregate shapes; they are not Cave Plan moves, savings estimates, implementation recipes, experiment eligibility, or proof that a code change is safe. Keep the workflow operator-chosen and evidence-first.

1. Read the exact observations

Require a logged-in Caveman CLI session and run:

caveman opportunities list

Read only the report_only_observations array. Do not select from the lifecycle data array. Preserve each server-provided title and observation verbatim. Handle these exact repository-profile ids:

  • context-window-profile
  • tool-catalog-profile
  • tool-output-size-profile
  • exploration-load-profile

These profiles have an immutable zero band and no actuation path. Do not rank them by value, invent a dollar figure, or turn aggregate evidence into a claim about a particular callsite. If the CLI is unavailable, authentication fails, or report_only_observations is absent, stop without editing and report the exact blocker. Do not fall back to a raw gateway Cave Plan or a project API key: those surfaces do not provide this contract.

Never select or apply these retired ids:

  • context-window-bloat
  • tool-catalog-utilization
  • verbose-tool-output

Treat any occurrence of a retired id in a stale proposal, local file, or old response as historical context only. Never revive its money, recipe, or lifecycle claim. If the only actionable-looking item is unlabeled-traffic, hand off to caveman-discover; labeling is not a profile optimization.

2. Ask the operator to choose

Present the available supported observations without ranking them. Include the id, the exact title, the exact observation, and last_seen_at. Ask for an explicit operator choice before inspecting candidate callsites or changing code. If no supported current observation exists, stop with no edit.

Read the whole SKILL.md on GitHub

More in Data and ML