GPU Cluster Evaluation
Evaluates an ML GPU cluster with FLOPS, bandwidth, and disk tests and writes a report.
Install
npx skills add stas00/ml-engineering --skill evaluate-cluster -g -a claude-code -y- Repository
- stas00/ml-engineering19.6k
- Category
- DevOps
- Updated
- 9d ago
- License
- CC-BY-SA-4.0
- Context cost
- Mediumabout 3,000 tokens
- Runs scripts
- No, instructions only
What it does
Evaluates an ML GPU cluster for a cloud trial or acceptance test: environment dump, isolated newest PyTorch, matmul FLOPS (MAMF/MSMF) on every GPU while the others compute, intra-node all-reduce bandwidth and per-call latency of every collective, the same inter-node on every node you were given (omit those sections if there is only one node), fio on local disk and shared FS, dated markdown report. Use when the user asks to evaluate a cluster, kick the tires on trial nodes, run cluster acceptance, or measure GPU/network/storage. Canonical copy: https://github.com/stas00/ml-engineering/blob/master/skills/evaluate-cluster/SKILL.md
Evaluate a GPU cluster
Always use the GitHub copy if it differs from this file. People pass around downloads that can be months or years old.
- This skill: https://github.com/stas00/ml-engineering/blob/master/skills/evaluate-cluster/SKILL.md
- Raw: https://raw.githubusercontent.com/stas00/ml-engineering/master/skills/evaluate-cluster/SKILL.md
Before doing any eval work, curl the raw URL. If it is not byte-identical to the file you are reading, stop following this copy and follow the GitHub file instead (fetch it, then continue from that text). If GitHub is unreachable, say so and continue with this file.
This file is the whole runbook. You do not need any other document from the author's books. Scripts and spec tables are on GitHub; fetch them (commands below). Optional background: How to evaluate the cluster.
Keep this skill up to date
Every correction the user makes to a report is a correction to this skill. When they rename a heading, reject a phrase, ask for a different TOC label, or point out an assumption, fix the report and edit this file in the same turn so the next eval starts from the corrected rule. A fix that lives only in one report is a fix you will be asked for again.
- Write the rule where the next agent will hit it, the section that produces that text, not a footnote.
- Encode the general principle, not just the one string. "Don't write
Skip reasons: none" became "never assert an absence". - When the correction changes a heading, update the heading rule, the TOC example, and every cross-reference to the old name in this file.
- Do not report the edit as done until you have grepped this file for the old wording.
Reports under reports/ other than the one you are writing are historical. Leave them alone.
