Trace QA

Analyze agent execution traces to answer questions about what happened, why it failed, how efficient it was, or any other aspect of the run.

Workflow

Always start with overview to understand the trace before diving into details.

1. Get the overview first

python scripts/fetch_trace.py <trace_id> overview

This returns metadata (status, duration, tokens, model) and summaries (request, answer preview, tool usage counts). Use this to orient yourself before going deeper.

2. Explore steps or LLM calls as needed

Depending on the user's question, drill into the relevant data:

User wants to know...	Command
What tools were called and in what order	steps [start] [count]
Full input/output of a specific tool call	step <N>
How many LLM calls and their token costs	llm-calls [start] [count]
What messages were sent to Claude in a specific turn	llm-call <N>
Just the final result	answer

3. Handle long content with segmented reads

When content is large, the script automatically segments output to ~4000 characters. If you see a [CONTINUED: ...] message at the end of output, call the command shown in that message to read the next segment. Repeat until all content is read.

Example sequence:

python scripts/fetch_trace.py <id> step 5
# Output ends with: [CONTINUED: use 'step 5 --offset 4000' for next segment]

python scripts/fetch_trace.py <id> step 5 --offset 4000
# Output ends with: [CONTINUED: use 'step 5 --offset 8000' for next segment]

python scripts/fetch_trace.py <id> step 5 --offset 8000
# Full content now read

Command Reference