# Evaluate Skill

Test a candidate OpenSEO skill end to end by running fresh, isolated Codex sessions...

- Repository: https://github.com/every-app/open-seo (25988 stars)
- Category: Testing
- License: MIT
- Last updated: 2026-10-31
- Runs scripts: yes, review before use

## Install

Claude Code: `npx skills add every-app/open-seo --skill evaluate-skill -g -a claude-code -y`
Codex: `npx skills add every-app/open-seo --skill evaluate-skill -g -a codex -y`
Cursor: `npx skills add every-app/open-seo --skill evaluate-skill -g -a cursor -y`
Gemini CLI: `npx skills add every-app/open-seo --skill evaluate-skill -g -a gemini-cli -y`
Claude Code plugin marketplace: `/plugin marketplace add every-app/open-seo`

## Description

Test a candidate OpenSEO skill end to end by running fresh, isolated Codex sessions against the local backend and scoring the reports they save. Use when editing a product skill (seo-audit, keyword-research, ...) and you need evidence that the new instructions produce better output, not just a cleaner file.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Evaluate a skill

A skill edit is only as good as the reports it produces. This workflow runs the candidate skill in sessions that know nothing except the skill file, a one-line business brief, and a locked-down local MCP endpoint, then compares what came back.

## Prerequisites

- Local server in no-auth mode: `AUTH_MODE=local_noauth` in `.env.local`, then `pnpm db:migrate:local` and `pnpm dev` (or `pnpm dev:agents`). Pass the URL it prints, with `/mcp` appended, as `--endpoint`. Default local D1, never remote bindings or remote Postgres. Keep it private; it has no login.
- `codex` CLI on the path. Runs use its configured default model at high reasoning; record the resolved model from the events log if you need to name it.
- Research credits: the local server still calls DataForSEO. An audit run makes 20 to 40 tool calls. Do not run more sessions than the comparison needs.

## Run

```sh
node .agents/skills/evaluate-skill/scripts/run.mjs \
  --skill seo-audit \
  --site example.com --site holdout.example \
  --brief "Evaluation brief: Acme at example.com sells <what>. The goal is relevant organic visitors who could become customers. No first-party analytics are connected." \
  --brief "Evaluation brief: <the holdout business>. ..." \
  --endpoint http://<local-dev-host>/mcp
```

Runs take about 10 to 14 minutes and execute in parallel. Launch the script detached, because a tool-call timeout that kills the shell kills the sessions too:

```sh
(nohup node .agents/skills/evaluate-skill/scripts/run.mjs ... > .logs/eval.log 2>&1 &)
```

What the script does, per site:

The rest: https://github.com/every-app/open-seo/blob/HEAD/.agents/skills/evaluate-skill/SKILL.md