
Codex Ab
Run an A/B codex review experiment, holistic codex review vs 3 focused dimension...
Install
npx skills add oliver-kriska/claude-elixir-phoenix --skill codex-ab -g -a claude-code -y- Repository
- oliver-kriska/claude-elixir-phoenix572
- Category
- Security
- Updated
- 5d ago
- License
- MIT
- Context cost
- Lightabout 895 tokens
- Runs scripts
- Yes, review before use
What it does
Run an A/B codex review experiment, holistic codex review vs 3 focused dimension passes (security, ecto, liveview) on the branch diff, classify findings, report a panel-value verdict. Use when the branch is fresh, before any codex review runs.
Codex Panel A/B (contributor instrument, verdict decided)
Answer one question with evidence: do dimension-focused codex passes find
real issues that one holistic codex exec review misses? Runs both on the
same diff, then classifies every focused finding against the holistic pass.
DECIDED 2026-07-10 after 4 runs (2 fresh): panel KILLED. Fresh-only 1 real miss / 1 false positive plus one zero-value run at 4× cost, > real misses did not outnumber FPs. Kept as contributor tooling (NOT distributed) for one possible retest: a UI-heavy diff with a single extra liveview-focused pass (2× cost). Scoreboard:
.claude/research/2026-07-03-codex-review-integration.md§7.
Usage
/codex-ab # A/B against main (~5 min, 4 codex runs)
/codex-ab develop # explicit base branch
Iron Laws
- FRESH DIFF ONLY, ask the user to confirm this branch has NOT been
codex-reviewed yet (cloud or
/phx:codex-loop). A drained diff returns NO FINDINGS everywhere and proves nothing, wasted quota - Verify every REAL MISS in the code before counting it, a focused finding only scores if the issue actually exists at that file:line
- Read ONLY the findings
.mdfiles, streams are diverted to.logfiles; never cat a log into context (10k+ lines each) - Exactly 4 codex runs, never re-run dimensions, bounded quota
- Persist the verdict, an unrecorded experiment is wasted quota
Workflow
Step 1: Preflight
Run command -v codex, missing → STOP with install hint. Then:
git status --shortdirty → warn (codex flags local dirt as findings)- Ask: "Has codex already reviewed this branch (PR review or codex-loop)?" If yes → STOP, explain the fresh-diff requirement (Iron Law 1)
Step 2: Run the A/B (background, ~5 min)
bash ${CLAUDE_SKILL_DIR}/scripts/codex-panel-ab.sh {base} \
.claude/reviews/codex-ab-$(date +%Y-%m-%d-%H%M)

