Benchmark Paper Template

Overview

A Benchmark paper does not win by proposing a new algorithm. It wins by defining a new evaluation dimension and shipping a construction pipeline that makes the measurement high-quality, scalable, and reproducible. This skill scaffolds the five pillars a reviewer checks, then gives you a six-part Introduction chain, a Section 2-7 skeleton, and a pre-submission checklist. Stage-specific depth lives in seven reference files under references/.

Core capabilities

1. Five-pillar completeness audit: is the Research Gap articulated? Is the Construction Pipeline principled? Is the Evaluation Framework fine-grained? Do the Empirical Findings reveal capability boundaries? Is a Companion Method warranted?
2. Introduction six-part logic chain: Background + Running Example, Existing-Benchmark Limitations (no more than three), Research Questions, Design Considerations, Our Proposal, Contributions.
3. Section skeleton for §2 to §7: Task and Design Goals, Construction Pipeline, Optional Companion Method, Experiments organized by RQ, Discussion and Research Opportunities, Related Work with benchmark comparison table, Conclusion.
4. Pre-submission self-check: four-category reviewer checklist (Introduction, Benchmark section, Experiments, Overall) with Critical, Major, Minor severity.

Benchmark paper vs technical paper

Dimension	Technical paper	Benchmark paper
Main contribution	Novel algorithm or method	Novel evaluation dimension or dataset
Introduction axis	Key Idea or Mechanism	Evaluation Gap and Benchmark Design Rationale
Problem definition	One-sentence goal	The problem definition IS the contribution
Heaviest chapter	Method	Construction Pipeline + Evaluation Framework
Experiments purpose	Prove "my method beats baselines"	Reveal "where model capability boundaries sit"
Canonical Figure 1	Method framework diagram	Running example + pipeline diagram