# Agent Benchmark

Framework for measuring and tracking agent response quality over time.

- Repository: https://github.com/vibeeval/vibecosystem (541 stars)
- Category: AI and agents
- License: MIT
- Last updated: 2026-08-08
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add vibeeval/vibecosystem --skill agent-benchmark -g -a claude-code -y`
Codex: `npx skills add vibeeval/vibecosystem --skill agent-benchmark -g -a codex -y`
Cursor: `npx skills add vibeeval/vibecosystem --skill agent-benchmark -g -a cursor -y`
Gemini CLI: `npx skills add vibeeval/vibecosystem --skill agent-benchmark -g -a gemini-cli -y`

## Description

Framework for measuring and tracking agent response quality over time. Detects regressions before they reach production. Use when evaluating agent changes, auditing quality, or establishing performance baselines.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Agent Benchmark Framework

Without benchmarks, we cannot know whether agent changes improve or degrade quality. This skill defines how to measure, track, and protect agent performance.

## When to Activate

- Before and after modifying any agent definition file
- When adding a new skill that an agent depends on
- Periodic quality audits (weekly/monthly)
- When a user reports degraded agent output
- Before promoting an agent from experimental to production

## Core Concepts

### Why Benchmarks Matter

Agent quality degrades silently. A prompt tweak that improves one response can break ten others. Without a baseline to compare against, every change is a guess. Benchmarks make quality visible and regressions detectable.

### Benchmark Types

| Type | Scope | Cost | Frequency |
|------|-------|------|-----------|
| Prompt Benchmark | Single agent, single task | Low | Every agent change |
| Task Benchmark | End-to-end scenario | Medium | Feature changes |
| Regression Suite | All critical agents | High | Weekly / before release |

## Directory Structure

The rest: https://github.com/vibeeval/vibecosystem/blob/HEAD/skills/agent-benchmark/SKILL.md