Arize Prompt Optimization
Optimizes, improves, and debugs LLM prompts using production trace data, evaluations...
Install
npx skills add github/awesome-copilot --skill arize-prompt-optimization -g -a claude-code -y- Repository
- github/awesome-copilot40.7k
- Category
- AI and agents
- Updated
- 6d ago
- License
- MIT
- Context cost
- Mediumabout 3,000 tokens
- Runs scripts
- No, instructions only
What it does
Optimizes, improves, and debugs LLM prompts using production trace data, evaluations, and annotations. Extracts prompts from spans, gathers performance signal, and runs a data-driven optimization loop using the ax CLI. Use when the user mentions optimize prompt, improve prompt, make AI respond better, improve output quality, prompt engineering, prompt tuning, or system prompt improvement.
Arize Prompt Optimization Skill
SPACE, All--spaceflags and theARIZE_SPACEenv var accept a space name (e.g.,my-workspace) or a base64 space ID (e.g.,U3BhY2U6...). Find yours withax spaces list.
Concepts
Where Prompts Live in Trace Data
LLM applications emit spans following OpenInference semantic conventions. Prompts are stored in different span attributes depending on the span kind and instrumentation:
| Column | What it contains | When to use |
|---|---|---|
attributes.llm.input_messages | Structured chat messages (system, user, assistant, tool) in role-based format | Primary source for chat-based LLM prompts |
attributes.llm.input_messages.roles | Array of roles: system, user, assistant, tool | Extract individual message roles |
attributes.llm.input_messages.contents | Array of message content strings | Extract message text |
attributes.input.value | Serialized prompt or user question (generic, all span kinds) | Fallback when structured messages are not available |
attributes.llm.prompt_template.template | Template with {variable} placeholders (e.g., "Answer {question} using {context}") | When the app uses prompt templates |
attributes.llm.prompt_template.variables | Template variable values (JSON object) | See what values were substituted into the template |
attributes.output.value | Model response text | See what the LLM produced |
attributes.llm.output_messages | Structured model output (including tool calls) | Inspect tool-calling responses |
