# Agent Platform Eval Flywheel

Measures and improves the quality of AI models and agents on Google Cloud using the...

- Repository: https://github.com/google/skills (21764 stars)
- Category: Data and ML
- License: Apache-2.0
- Last updated: 2026-11-01
- Runs scripts: yes, review before use

## Install

Claude Code: `npx skills add google/skills --skill agent-platform-eval-flywheel -g -a claude-code -y`
Codex: `npx skills add google/skills --skill agent-platform-eval-flywheel -g -a codex -y`
Cursor: `npx skills add google/skills --skill agent-platform-eval-flywheel -g -a cursor -y`
Gemini CLI: `npx skills add google/skills --skill agent-platform-eval-flywheel -g -a gemini-cli -y`
Claude Code plugin marketplace: `/plugin marketplace add google/skills`

## Description

Measures and improves the quality of AI models and agents on Google Cloud using the Eval Quality Flywheel methodology. Use when generating synthetic user scenarios, evaluating an agent or model, building an eval dataset, picking or writing evaluation metrics, analyzing failures, comparing results before and after a fix, or when guidance is needed on Agent Platform eval methodology, including dataset schema, LLM-as-judge scoring, and common failure causes. For fine-tuning, use agent-platform-tuning. For general production deployment, use agent-platform-deploy.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Agent Platform Eval Flywheel Skill

Help users evaluate and iteratively improve GenAI models and agents using the
Agent Platform GenAI Evaluation SDK (`google.genai` / `agentplatform`).

## When to use this skill

-   Evaluating GenAI agents or models with the Agent Platform GenAI Evaluation
    SDK (`client.evals.evaluate()`).
-   Creating evaluation datasets from session traces, pandas DataFrames, or
    synthetic generation.
-   Selecting, configuring, or writing custom evaluation metrics.
-   Analyzing rubric verdicts, loss patterns, and clustering failures.
-   Suggesting concrete code/prompt improvements based on eval results.
-   Evaluating a model served on an Agent Platform **endpoint** (BYOM) or a
    **Model-as-a-Service (MaaS)** model by ID, including deploying the model
    first if needed. For this case, follow
    [references/deployment.md](references/deployment.md) and use the
    `endpoint_evaluation.py` / `maas_evaluation.py` scripts.

## Safety & Confirmation Tiers (CRITICAL)

Before executing any commands or scripts on behalf of the user, you MUST adhere
to the following safety tiers based on the action requested:

The rest: https://github.com/google/skills/blob/HEAD/skills/cloud/agent-platform-eval-flywheel/SKILL.md