Regression Alert

Detect whether Claude Code reliability is getting worse period-over-period, and optionally arm an alert so it never has to be checked by hand again. Scope is reliability/failures only, for cache/cost/compaction drift, use ccam-insights' regression-watch instead.

Input

The user provides: $ARGUMENTS

This may be:

• empty or "all", check error rate and tool-failure rate (default)
• "errors", APIError-rate regression only
• "tools", tool-failure-rate regression only
• a window like "7 vs 7" or "30 vs 30", recent vs baseline window sizes (default: last 7 days vs the prior 7)
• "arm", after reporting, also create an alert rule via POST /api/alerts/rules (only on explicit request)

Data Sources

Endpoint	Returns
GET /api/analytics	daily_events (365d), daily_sessions (365d), event_types, split into recent vs baseline windows to compute per-window failure rates
GET /api/events?session_id=X	Per-session stream, localize a regression to the sessions driving it
GET /api/alerts/rules	Existing alert rules, check whether a matching reliability rule already exists before arming a new one
POST /api/alerts/rules	Create a new alert rule (only when the user says "arm")

Report Sections

1. Windowing

Split history into a recent window (newer) and a baseline window (the equal-length period just before it). Default: recent = last 7 days, baseline = the prior 7. Use daily_events/daily_sessions to bucket counts by day.

2. Error-Rate Regression

• Per window: error rate = APIError count / total events.
• Compare recent vs baseline. Flag if recent is higher. Report absolute change (pp) and relative change (%), plus the recent sessions contributing the most APIError events.