# Agent Evaluation

Evaluate agent behavior with versioned cases and explicit verifiers.

- Repository: https://github.com/sickn33/agentic-awesome-skills (48984 stars)
- Category: Testing
- License: MIT
- Last updated: 2026-11-01
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add sickn33/agentic-awesome-skills --skill agent-evaluation -g -a claude-code -y`
Codex: `npx skills add sickn33/agentic-awesome-skills --skill agent-evaluation -g -a codex -y`
Cursor: `npx skills add sickn33/agentic-awesome-skills --skill agent-evaluation -g -a cursor -y`
Gemini CLI: `npx skills add sickn33/agentic-awesome-skills --skill agent-evaluation -g -a gemini-cli -y`
Claude Code plugin marketplace: `/plugin marketplace add sickn33/agentic-awesome-skills`

## Description

Evaluate agent behavior with versioned cases and explicit verifiers. Use when comparing agent or prompt changes, reproducing failures, or running agent regression tests.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Agent Evaluation

Evaluate observable agent behavior against task-specific cases. Modified by AAS maintainers on 2026-09-05 to remove unsupported benchmark claims, correct uncertainty/error reporting and separate optional architecture sketches from the operating procedure.

## When to Use

Use when comparing a changed agent, prompt or tool configuration, reproducing an observed failure, or estimating reliability on a declared task distribution. Do not infer product readiness from a public benchmark percentage or a generic score threshold.

## Prerequisites

- A versioned case set with expected observable outcomes and permission boundaries.
- A known baseline and candidate revision, including model, prompt, tools, configuration and runtime versions.
- Authorized synthetic or redacted inputs, isolated targets and a bounded token, time and cost budget.
- A verifier that distinguishes wrong outcomes, expected safe rejections, evaluator failures and infrastructure outages. Provider access is needed only if the declared evaluation calls that provider.

## Evaluation procedure

The rest: https://github.com/sickn33/agentic-awesome-skills/blob/HEAD/plugins/agentic-awesome-skills-claude/skills/agent-evaluation/SKILL.md