Skills

Live Mastracode Instrumentation

Reproduce and measure a Mastra Code runtime bug against a real model by driving the...

Install

npx skills add mastra-ai/mastra --skill live-mastracode-instrumentation -g -a claude-code -y
Repository
mastra-ai/mastra29.3k
Category
Testing
Updated
3d ago
License
See repository
Context cost
Mediumabout 1,840 tokens
Runs scripts
No, instructions only

What it does

Reproduce and measure a Mastra Code runtime bug against a real model by driving the built TUI headlessly in tmux while every layer (network, provider stream, run engine, TUI) appends timestamped JSONL. Use when a bug depends on real provider timing, streaming order, or event flow that mocked tests and fixtures may not reproduce, e.g. wrong token rates, flicker, stuck status, ordering races, or "only happens sometimes with a real model".

Live Mastra Code instrumentation

Run the real TUI against a real model, have an agent type the prompts, and log timestamps at each layer. Comparing the layers step by step shows where timing or ordering goes wrong: at the provider, in the network wrapper, in stream conversion, in the run engine, or in the UI reducer.

Pair this with debugging-difficult-bugs: instrument, reproduce, read the log before fixing, and clean up afterwards. Afterwards, turn the finding into deterministic checked-in coverage (unit tests plus a TUI E2E scenario under mastracode/tui/e2e/tui/). A live run is evidence, not a regression test.

Cost and consent

The TUI uses the user's configured model and stored credentials (for example a Claude Max OAuth login), so every prompt spends their quota. Use short prompts. Tell the user which model you ran and roughly how many sessions.

1. Add temporary diagnostics

Append one JSON line per event to <cwd>/debug-token-rate.jsonl (or a similarly named file) with appendFileSync. Rules:

  • Metadata only: timestamps, event types, ids, lengths, and token counts. Never log prompt, response, or tool content.
  • Tag every record with a unique build marker (for example 'my-bug-diagnostics-v1'). That lets you confirm the build you ran contains the instrumentation and find every leftover when cleaning up.
  • Wrap each append in try {} catch {} so logging can never break the run.
  • Take the timestamp before writing. A synchronous append adds a little latency to the path you are measuring, which is fine for millisecond-scale timing. If you are measuring anything finer, push records onto an in-memory array and write them out once when the run ends.
  • Log at the earliest and latest point of each layer you care about, so the timestamps can be lined up per step.

Useful hook points, from outermost to innermost:

Read the whole SKILL.md on GitHub

More in Testing