Tool Prompt Optimization
Optimize the description prompts an AI agent reads to learn its built-in tools (the...
Install
npx skills add can1357/oh-my-pi --skill tool-prompt-optimization -g -a claude-code -y- Repository
- can1357/oh-my-pi37.4k
- Category
- Testing
- Updated
- 3d ago
- License
- MIT
- Context cost
- Mediumabout 1,628 tokens
- Runs scripts
- Yes, review before use
What it does
Optimize the description prompts an AI agent reads to learn its built-in tools (the `.md` files under prompts/tools/). Two halves: (1) measure how much of a prompt is already inferable from the tool's JSON parameter schema + name, to prune redundancy with evidence; (2) house authoring rules for what belongs in a tool prompt vs what stays in code. Use when auditing, trimming, writing, or reviewing tool prompts, deciding what schema field descriptions already cover, or testing schema-vs-prompt overlap before deleting prompt lines.
Tool Prompt Optimization
Prompt/schema overlap: content reconstructible from (name, JSON schema, blank outline) is a prune candidate, never an automatic delete. Probe this overlap for evidence, not vibes: predict the prompt body from those inputs. Reliably recovered lines: candidates; no-model recovery: load-bearing, keep.
Run probe
scripts/probe.ts: @oh-my-pi/pi-ai completeSimple; production-matching model/auth/provider behavior.
bun .omp/skills/tool-prompt-optimization/scripts/probe.ts \
--schema <file|json> --template <file|text> --name <tool_name>
- Required:
--schema,--template, file path or inline value. - No
--model: panelfireworks/kimi-k2.7-code,anthropic/claude-opus-4-8,openai/gpt-5.5×--samples(default 3); requiresFIREWORKS_API_KEY/ANTHROPIC_API_KEY/OPENAI_API_KEY. --model p/id,p/id: override panel. Tune:--samples N,--max-tokens,--json.- Programmatic:
import { probe } from "./scripts/probe.ts"→{ prompt, results: [{ model, samples: [{ text, stopReason, usage, error }] }] }.
Builtin shortcut, preferred for this repo
scripts/probe-builtin.ts instantiates the live tool; gets exact toolWireSchema, tool.description, and derived outline:
bun .omp/skills/tool-prompt-optimization/scripts/probe-builtin.ts --tool <name> [--no-summary] [--show]
--show: resolved schema, derived outline, real prompt; exits without API calls. Inspect before spending tokens.--no-summary: direct summary-line-blank ablation.--samples/--model/--max-tokens/--json: panel passthrough. Output ends with real prompt for in-place diff.- Factory-map bypasses settings allowlist: gated
irc,github, … resolve. Construction availability gate (e.g. missingghCLI) → manual inputs.
Inputs
Schema: wire schema the model sees, never hand-sketch. Arktype:
