# Analyze Exports

Analyze tau2-bench task export files to identify exactly where and why tasks failed.

- Repository: https://github.com/Leanmcp/gateway-skills (2122 stars)
- Category: Testing
- License: MIT
- Last updated: 2026-10-07
- Runs scripts: yes, review before use

## Install

Claude Code: `npx skills add Leanmcp/gateway-skills --skill analyze-exports -g -a claude-code -y`
Codex: `npx skills add Leanmcp/gateway-skills --skill analyze-exports -g -a codex -y`
Cursor: `npx skills add Leanmcp/gateway-skills --skill analyze-exports -g -a cursor -y`
Gemini CLI: `npx skills add Leanmcp/gateway-skills --skill analyze-exports -g -a gemini-cli -y`

## Description

Analyze tau2-bench task export files to identify exactly where and why tasks failed. Map failures to specific conversation rounds, show root causes, check whether relevant documentation was available, and cite sources. Use this skill whenever you need to debug a tau2-bench evaluation, understand why an agent made wrong decisions, identify missing knowledge, or review task performance, especially when context length, eligibility criteria, or policy compliance might be at fault.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Analyze Exports Skill

## Purpose

When a tau2-bench task returns reward 0.0 (or fails), the export file contains everything you need to understand why, but it's buried in tool-call summaries and conversation transcripts. This skill systematically analyzes the export to show:

1. **Where it failed**, the exact round (Assistant #X, User #Y) and tool-call sequence
2. **What should have happened**, ground truth from the task explanation
3. **Why it failed**, root cause (missing logic, wrong arguments, bad order, policy misunderstanding)
4. **Was the docs available?**, whether the KB had the answer the agent needed
5. **Evidence**, quotes and citations from both the transcript and source documents

## Input Format

An export file (plain text) containing:
- `RUN:` and `TASK:` metadata
- `★ TASK EXPLANATION` (success criteria, expected actions)
- `🛠 TOOLS LOADED` (available tools and KB docs)
- `⇄ TOOL-CALL SUMMARY` (OURS vs GROUND TRUTH comparison)
- Full conversation transcript with tool calls and results

## Output Format & File Organization

Analysis reports are saved in a standardized folder structure for easy organization and review:

```
FAILURE_ANALYSIS/
├── analysis/
│   ├── hard_15/
│   │   ├── task_037_detailed_analysis.md
│   │   ├── task_038_detailed_analysis.md
│   │   ├── task_039_detailed_analysis.md
│   │   └── task_041_detailed_analysis.md
│   ├── hard_16/
│   │   ├── task_053_detailed_analysis.md
│   │   └── task_054_detailed_analysis.md
│   └── [other_categories]/
│       └── [task_id]_detailed_analysis.md
└── README.md (overview of all analyses)
```

**File Naming Convention**: `[task_id]_detailed_analysis.md`

**Folder Organization**: Group by difficulty level (hard_15, hard_16, etc.) or category to match your existing structure.

**Each analysis file contains these sections:**

### Report Structure

You will receive a structured analysis report with these sections:

```
## Export Analysis: [TASK_ID]

The rest: https://github.com/Leanmcp/gateway-skills/blob/HEAD/skills/analyze-exports/SKILL.md