# Creating MCP Agent Evals

Use when adding Langfuse MCP agent evals for a tool family of the Apify MCP server...

- Repository: https://github.com/apify/apify-mcp-server (14245 stars)
- Category: AI and agents
- License: MIT
- Last updated: 2026-10-31
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add apify/apify-mcp-server --skill creating-mcp-agent-evals -g -a claude-code -y`
Codex: `npx skills add apify/apify-mcp-server --skill creating-mcp-agent-evals -g -a codex -y`
Cursor: `npx skills add apify/apify-mcp-server --skill creating-mcp-agent-evals -g -a cursor -y`
Gemini CLI: `npx skills add apify/apify-mcp-server --skill creating-mcp-agent-evals -g -a gemini-cli -y`

## Description

Use when adding Langfuse MCP agent evals for a tool family of the Apify MCP server ("create evals for the storage tools"), when eval cases fail and you must decide whether the case, the tool, or its description is at fault, or when eval runs show tool errors in Langfuse traces.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Creating MCP agent evals for an MCP tool family

## Overview

Build a small, calibrated Langfuse eval suite for one tool family (tasks, storage, runs, …), then use its failures to fix the tools. Core principle: **evals are designed from user intent, never from tool descriptions**, the eval defines what should work; descriptions get fixed afterward to make naive agents pass it.

Commands, item shapes, probe patterns, and sweep queries: [reference.md](reference.md).

## The flow

The rest: https://github.com/apify/apify-mcp-server/blob/HEAD/.claude/skills/creating-mcp-agent-evals/SKILL.md