# Benchmark Paper Template

Structures benchmark papers into five pillars with a section skeleton and checklist.

- Repository: https://github.com/HKUSTDial/Supervisor-Skills (11333 stars)
- Category: Writing
- License: CC-BY-4.0
- Last updated: 2026-10-16
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add HKUSTDial/Supervisor-Skills --skill benchmark-paper-template -g -a claude-code -y`
Codex: `npx skills add HKUSTDial/Supervisor-Skills --skill benchmark-paper-template -g -a codex -y`
Cursor: `npx skills add HKUSTDial/Supervisor-Skills --skill benchmark-paper-template -g -a cursor -y`
Gemini CLI: `npx skills add HKUSTDial/Supervisor-Skills --skill benchmark-paper-template -g -a gemini-cli -y`

## Description

Structures Benchmark and Evaluation papers using the five-pillar framework (Research Gap, Construction Pipeline, Evaluation Framework, Empirical Findings, optional Companion Method). Returns a completeness audit, a six-part Introduction logic chain, a Section 2-7 skeleton, and a pre-submission checklist. Use when writing a benchmark paper, structuring a benchmark paper, checking whether a benchmark idea is substantive, drafting a benchmark Introduction, or planning the data-construction pipeline or experiments.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Benchmark Paper Template

## Overview

A Benchmark paper does not win by proposing a new algorithm. It wins by defining a new evaluation dimension and shipping a construction pipeline that makes the measurement high-quality, scalable, and reproducible. This skill scaffolds the five pillars a reviewer checks, then gives you a six-part Introduction chain, a Section 2-7 skeleton, and a pre-submission checklist. Stage-specific depth lives in seven reference files under `references/`.

## Core capabilities

1. **Five-pillar completeness audit**: is the Research Gap articulated? Is the Construction Pipeline principled? Is the Evaluation Framework fine-grained? Do the Empirical Findings reveal capability boundaries? Is a Companion Method warranted?
2. **Introduction six-part logic chain**: Background + Running Example, Existing-Benchmark Limitations (no more than three), Research Questions, Design Considerations, Our Proposal, Contributions.
3. **Section skeleton for §2 to §7**: Task and Design Goals, Construction Pipeline, Optional Companion Method, Experiments organized by RQ, Discussion and Research Opportunities, Related Work with benchmark comparison table, Conclusion.
4. **Pre-submission self-check**: four-category reviewer checklist (Introduction, Benchmark section, Experiments, Overall) with Critical, Major, Minor severity.

## Benchmark paper vs technical paper

| Dimension | Technical paper | Benchmark paper |
|---|---|---|
| Main contribution | Novel algorithm or method | Novel evaluation dimension or dataset |
| Introduction axis | Key Idea or Mechanism | Evaluation Gap and Benchmark Design Rationale |
| Problem definition | One-sentence goal | The problem definition IS the contribution |
| Heaviest chapter | Method | Construction Pipeline + Evaluation Framework |
| Experiments purpose | Prove "my method beats baselines" | Reveal "where model capability boundaries sit" |
| Canonical Figure 1 | Method framework diagram | Running example + pipeline diagram |

The rest: https://github.com/HKUSTDial/Supervisor-Skills/blob/HEAD/skills/benchmark-paper-template/SKILL.md