Manage eval-gated experiments

Treat every lifecycle change as a production control action. Read current state and results, then report one supported recommendation or block. Current agent MCP is intentionally read-only: control-api does not yet enforce a complete lifecycle transition table and evidence gate atomically.

Non-negotiable gates

1. A request to review, inspect, explain, or recommend authorizes reads only.
2. Never approve an experiment whose results are pending, whose required guardrails are absent, or whose evidence reports a breach.
3. Never convert experiment lift into verified_savings. Only active real traffic plus provider-causal, provider-complete ledger evidence can do that.
4. Never supply an organization id. Project and tenant scope come from the logged-in Caveman identity and server RBAC.
5. Never execute a lifecycle mutation, even after user approval. Exact <action>:<experiment_id> strings are agent-generatable and are not proof of human intent.
6. Unknown states and server errors fail closed. Report exact cave_snake_code.

Step 1, Load project and experiment

Prefer MCP:

caveman_context {}
caveman_experiment_get {"action":"get","experiment_id":"<id>"}
caveman_experiment_get {"action":"results","experiment_id":"<id>"}

Use {"action":"list"} when the user has not named an id.

CLI fallback:

caveman cloud experiments list
caveman cloud experiments show <id>
caveman cloud experiments results <id>

Stop if login, project, experiment, or results are unavailable.

Step 2, Evaluate evidence

Report:

• current lifecycle state and safety class;
• control and candidate sample sizes;
• quality or eval result;
• latency, error, cost, retry, drop, and escalation guardrails when present;
• evidence cost;
• rollback or hold reason;
• whether result is pending, failed, promotable, or active.