# Backtest Diagnose

Diagnose failed or underperforming backtests, locate the root cause, and fix the issue

- Repository: https://github.com/HKUDS/Vibe-Trading (37375 stars)
- Category: Engineering
- License: MIT
- Last updated: 2026-11-01
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add HKUDS/Vibe-Trading --skill backtest-diagnose -g -a claude-code -y`
Codex: `npx skills add HKUDS/Vibe-Trading --skill backtest-diagnose -g -a codex -y`
Cursor: `npx skills add HKUDS/Vibe-Trading --skill backtest-diagnose -g -a cursor -y`
Gemini CLI: `npx skills add HKUDS/Vibe-Trading --skill backtest-diagnose -g -a gemini-cli -y`

## Description

Diagnose failed or underperforming backtests, locate the root cause, and fix the issue

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Backtest Diagnosis

## Overview

Use this skill when a user reports that a backtest failed, raised an error, or produced poor results.

## Diagnostic Workflow

1. **Read existing artifacts**: use `read_file` to inspect `artifacts/metrics.csv`, `equity.csv`, and `trades.csv`
2. **Read the code**: use `read_file` to inspect `code/signal_engine.py` and `config.json`
3. **Classify the issue**: determine the root cause using the error taxonomy below
4. **Apply the fix**: use `edit_file` to modify the code, then rerun the backtest
5. **Verify the fix**: use `read_file` to inspect the new `metrics.csv`

## Error Taxonomy

### Runtime Errors (`exit_code != 0`)

| Error Type | Common Cause | Fix |
|---------|---------|---------|
| ImportError | Missing dependency | `bash("pip install xxx")` |
| KeyError | DataFrame column-name mismatch | Check the actual column names in `data_map` |
| IndexError | Empty data or insufficient length | Add length checks |
| TypeError | Incorrect signal type | Ensure the return value is `pd.Series` |

### Logic Bugs (Backtest Succeeds but Results Are Abnormal)

1. **Zero trades** (`trade_count=0`): signal-logic bug. Conditions are too strict, so the signal stays at 0. Check whether entry and exit logic is reasonable, and inspect the signal series to confirm it is not all zeros.
2. **Late trades** (first trade occurs more than 2 years after the backtest start): data-filtering bug. The lookback window may be too long, or the initial data segment may have been dropped. Shorten the window or check whether `dropna` is too aggressive.
3. **Capital utilization < 50%** (mostly in cash): position-sizing bug. Signal triggers may be too sparse, or the position-sizing logic may be wrong.
4. **Open position at the end** (a position still exists when the backtest ends): exit-timing bug. Forced liquidation may be missing, or exit logic does not cover the final segment.

### Data Errors

The rest: https://github.com/HKUDS/Vibe-Trading/blob/HEAD/agent/src/skills/backtest-diagnose/SKILL.md