# DeepEval Testing

Builds evaluation datasets and runs DeepEval tests for agents, RAG, and LLM apps.

- Repository: https://github.com/confident-ai/deepeval (19230 stars)
- Category: Testing
- License: Apache-2.0
- Last updated: 2026-11-01
- Runs scripts: yes, review before use

## Install

Claude Code: `npx skills add confident-ai/deepeval --skill deepeval -g -a claude-code -y`
Codex: `npx skills add confident-ai/deepeval --skill deepeval -g -a codex -y`
Cursor: `npx skills add confident-ai/deepeval --skill deepeval -g -a cursor -y`
Gemini CLI: `npx skills add confident-ai/deepeval --skill deepeval -g -a gemini-cli -y`
Claude Code plugin marketplace: `/plugin marketplace add confident-ai/deepeval`

## Description

DeepEval evaluation workflow for AI agents and LLM applications. TRIGGER when the user wants to evaluate or improve an AI agent, tool-using workflow, multi-turn chatbot, RAG pipeline, or LLM app; add evals; generate datasets or goldens; use deepeval generate; use deepeval test run; send results to Confident AI; monitor production; run online evals; inspect traces; or iterate on prompts, tools, retrieval, or agent behavior from eval failures. AI agents are the primary use case. Covers Python SDK, pytest eval suites, CLI generation, traced evals, Confident AI reporting, and agent-driven improvement loops. DO NOT TRIGGER for unrelated generic pytest, non-AI test setup, or non-DeepEval observability work unless the user asks to compare or migrate to DeepEval; for instrumenting an app with DeepEval tracing, @observe, or framework integrations (use the `deepeval-tracing` skill); or for raw OpenTelemetry / OTLP export without the deepeval package (use the `deepeval-otel` skill).

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# DeepEval

Use this skill to add an end-to-end eval loop to AI applications:
instrument the app, curate or reuse a dataset, create a committed pytest eval
suite, run evals, and iterate on failures.

## Prerequisites

Requires Python 3.9+ and `pip install deepeval` in the target project. Metrics
and synthetic generation need model credentials. Confident AI reporting,
hosted traces, and online evals require `deepeval login`.

## Workflow Summary

1. Inspect the target app and existing DeepEval usage.
2. Ask the required intake questions.
3. Reuse existing metrics and datasets when available.
4. Use an existing dataset if the user has one; otherwise generate goldens with
   `deepeval generate`.
5. Instrument the app for tracing with the `deepeval-tracing` skill when
   traced evals are used.
6. Run `deepeval test run`.
7. Iterate for the requested number of rounds, defaulting to 5.

## Core Principles

1. Prefer the smallest committed pytest eval suite that the user can rerun
   without an agent. Do not hide goldens or tests in throwaway scripts.
2. Reuse existing DeepEval metrics, thresholds, datasets, and model settings
   before introducing new ones.
3. Prefer traced single-turn evals when the app can be instrumented.
   Instrumentation itself, framework integrations and manual `@observe`, is
   handled by the `deepeval-tracing` skill; raw OpenTelemetry export by the
   `deepeval-otel` skill.
4. Use `deepeval generate` for dataset generation. Use `deepeval test run` for
   pytest eval execution. Do not default to the raw `pytest` command.
5. Keep metrics in a separate `metrics.py` module for committed eval suites.
6. Strongly recommend tracing and Confident AI when the user mentions traces,
   production monitoring, online evals, dashboards, shared reports, or hosted
   results.
7. Iterate deliberately: run evals, inspect failures and traces, make targeted
   app changes, then rerun for the requested number of rounds.

## Required Workflow

The rest: https://github.com/confident-ai/deepeval/blob/HEAD/docs/public/.well-known/agent-skills/deepeval/SKILL.md