Acl Artifact Evaluation
Use when packaging code, datasets, prompts, model outputs, or annotation materials for...
Install
npx skills add brycewang-stanford/Awesome-Journal-Skills --skill acl-artifact-evaluation -g -a claude-code -y- Repository
- brycewang-stanford/Awesome-Journal-Skills1.3k
- Category
- Data and ML
- Updated
- 1mo ago
- License
- MIT
- Context cost
- Lightabout 1,356 tokens
- Runs scripts
- No, instructions only
What it does
Use when packaging code, datasets, prompts, model outputs, or annotation materials for an ACL submission under ACL Rolling Review, covering anonymized supplement archives, scientific-artifact items of the Responsible NLP checklist, licensing and intended-use documentation, data statements, and post-acceptance public release.
ACL Artifact Evaluation
Use this to plan the evidence package around an ACL paper. ACL has no separate artifact-badge track; instead, artifact scrutiny is folded into review through the supplement archive and Section B ("scientific artifacts") of the Responsible NLP checklist, which reviewers cross-check against the PDF.
What counts as an artifact here
- Code: training/inference scripts, evaluation harnesses, prompt templates.
- Data: new corpora, annotations, filtered subsets of existing corpora, test suites, adversarial sets.
- Model outputs: generations, ranked lists, logits used in analysis, often the cheapest way to make an LLM paper checkable without GPUs.
- Human-subject materials: annotation guidelines, interface screenshots, consent text, compensation description.
Submission-time packaging rules
- Supplements upload as .tgz/.zip through the OpenReview form; links to tracked cloud storage are not acceptable, and any linked page must be anonymous.
- Scrub identity everywhere reviewers can look: file paths, git metadata, notebook author fields, license headers, dataset hosting pages, README contact lines.
- Reviewers are not required to open supplements. The paper plus checklist must stand alone; the archive is for verification, not for essential content.
