# Add Runner Evaluation

Adds a runner protocol evaluation with its roster, assertions and report fixture.

- Repository: https://github.com/paperclipai/paperclip (109885 stars)
- Category: Testing
- License: MIT
- Last updated: 2026-11-01
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add paperclipai/paperclip --skill add-runner-eval -g -a claude-code -y`
Codex: `npx skills add paperclipai/paperclip --skill add-runner-eval -g -a codex -y`
Cursor: `npx skills add paperclipai/paperclip --skill add-runner-eval -g -a cursor -y`
Gemini CLI: `npx skills add paperclipai/paperclip --skill add-runner-eval -g -a gemini-cli -y`

## Description

Add or extend a Paperclip Runner protocol evaluation definition, roster, assertion, or report fixture with provenance and narrow validation.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Add a Runner Eval

Use this skill for the **Runner Evals** family: a real Runner/provider session
against a seeded mock control plane. Product browser/server/database/Daytona
coverage belongs in [add-product-e2e-eval](../add-product-e2e-eval/SKILL.md).

Locate the Paperclip checkout using `PAPERCLIP_ROOT` when supplied, or
`git rev-parse --show-toplevel` from a checkout. From outside Git, inspect the
workspace roots (for example `~/paperclipai/paperclip`) and verify that the
selected root contains `packages/paperclip-runner` and `tests/runner-e2e`.
Locate `paperclip-evals` using `PAPERCLIP_EVALS_ROOT` or a discovered sibling;
a worktree's parent directory need not contain that repository. Read
`doc/evals.md` and `packages/paperclip-runner/docs/runner-protocol-live-evals.md`,
then inspect the nearest existing case, roster, schema, and report test before
editing. Definitions and authored cases belong in the sibling
`paperclip-evals/evals/paperclip-runner`; Runner integration, aggregation,
viewer, and publication behavior belongs in `packages/paperclip-runner`.
Keep the control-plane boundary explicit in names and documentation.

The sibling eval README is the concrete map: cases live under `cases/`,
company fixtures under `fixtures/`, runtime/model settings under `configs/`,
selections under `rosters/`, and maintained campaign membership under
`campaigns/live-direct-full.json`. Update inventory/coverage mappings when the
program requires them; a new file alone does not join the maintained campaign.
From the Evals repository root, adapt these provider-free checks to the case
and roster you changed. Run the reliability-plan validator only when that
separate plan changes:

The rest: https://github.com/paperclipai/paperclip/blob/HEAD/.agents/skills/add-runner-eval/SKILL.md