# Agent Evaluation Reporting

Use when summarizing agent evaluations where autonomous, assisted, failed, timed-out...

- Repository: https://github.com/sickn33/agentic-awesome-skills (48984 stars)
- Category: AI and agents
- License: MIT
- Last updated: 2026-11-01
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add sickn33/agentic-awesome-skills --skill agent-evaluation-reporting -g -a claude-code -y`
Codex: `npx skills add sickn33/agentic-awesome-skills --skill agent-evaluation-reporting -g -a codex -y`
Cursor: `npx skills add sickn33/agentic-awesome-skills --skill agent-evaluation-reporting -g -a cursor -y`
Gemini CLI: `npx skills add sickn33/agentic-awesome-skills --skill agent-evaluation-reporting -g -a gemini-cli -y`
Claude Code plugin marketplace: `/plugin marketplace add sickn33/agentic-awesome-skills`

## Description

Use when summarizing agent evaluations where autonomous, assisted, failed, timed-out, or invalid outcomes must remain distinct and comparable.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Agent Evaluation Reporting

## Overview

Turn raw agent evaluation runs into a decision-ready report without hiding failures or overstating capability. Keep outcome populations, denominators, latency populations, and experiment conditions explicit so readers can reproduce every headline number.

## When to Use This Skill

- Use when reporting benchmark, regression, pilot, or production evaluation runs for an AI agent.
- Use when autonomous and human-assisted completions appear in the same result set.
- Use when failures, timeouts, infrastructure-invalid runs, retries, or partial results affect the denominator.
- Use when comparing two agents, prompts, harnesses, or releases and deciding whether the comparison is valid.

## How It Works

### Step 1: Freeze the comparison contract

Record the task set and sampling, model and provider, prompt or policy version, tool and harness versions, evaluator rubric, timeout and retry policy, token or cost budget, environment, and human-intervention policy. Assign the configuration a stable label or digest.

If a material condition differs between runs, mark the comparison as non-equivalent. Report a directional observation only; do not claim that the changed agent caused the difference.

### Step 2: Build a mutually exclusive outcome ledger

Classify every scheduled attempt exactly once:

| Outcome | Meaning |
|---|---|
| `autonomous_success` | The agent satisfied the evaluator without human intervention. |
| `assisted_success` | The task succeeded only after a human intervened. |
| `failure` | The run reached a terminal, evaluable failure. |
| `timeout` | The run exhausted its declared time or step budget. |
| `invalid` | The agent never received a valid evaluation because the harness, environment, or input failed. |

The rest: https://github.com/sickn33/agentic-awesome-skills/blob/HEAD/plugins/agentic-awesome-skills-claude/skills/agent-evaluation-reporting/SKILL.md