BAT Story Evaluation

You are the evaluator. Run the steps in order; each one uses the output of the step before it.

Parse Arguments

From $ARGUMENTS, extract:

• --baseline: REQUIRED. Git tag/branch of the released version (e.g., v6.6.1).
• --agents: Agent list (default: gemini). Comma-separated.
• --stories: Force specific pre-built story IDs (e.g., s01,s02). Overrides triage selection.
• --all-stories: Skip triage, run ALL pre-built stories.
• --keep-container: Keep HA containers alive after run for manual inspection.
• --model: Model for Claude agent (e.g., haiku, sonnet).

If $ARGUMENTS is --help or missing --baseline, show usage and stop:

/bat-story-eval --baseline v6.6.1
/bat-story-eval --baseline v6.6.1 --agents gemini,claude
/bat-story-eval --baseline v6.6.1 --stories s01,s02
/bat-story-eval --baseline v6.6.1 --all-stories --agents claude --model haiku

Step 0: Triage (Diff Analysis + Custom Story Design)

0a. Compute Diff

cd "$(dirname "$(git rev-parse --path-format=absolute --git-common-dir)")/worktree/uat-stories"
git diff <baseline>..HEAD -- src/ha_mcp/ --stat
git diff <baseline>..HEAD -- src/ha_mcp/ --name-only

Classify changed files:

• Tool modules (tools/tools_*.py): specific tool implementations changed
• Core code (client/, server.py, errors.py, tools/util_helpers.py): affects all tools
• Utilities (utils/, resources/): may affect all tools
• No src/ changes: only tests/docs/config, select 2 smoke-test stories

0b. Select Pre-built Stories

Skip if --stories or --all-stories was passed.