Profile Vllm Performance
Profile vLLM and RT-VLM inference to identify and remove GPU-idle gaps, underfilled...
Install
npx skills add NVIDIA-AI-Blueprints/video-search-and-summarization --skill profile-vllm-performance -g -a claude-code -y- Category
- Engineering
- Updated
- 4d ago
- License
- Apache-2.0
- Context cost
- Lightabout 1,336 tokens
- Runs scripts
- No, instructions only
What it does
Profile vLLM and RT-VLM inference to identify and remove GPU-idle gaps, underfilled batches, transfer stalls, serialized multimodal work, scheduler gaps, or KV pressure. Use this skill when GPU utilization is unexpectedly low or bursty, capacity trails another runtime, TTFT or ITL regresses, or multimodal prefill and decode appear serialized. Not for running a first benchmark without a reproducible fixed-load signal.
Profile vLLM Performance
Find the dominant loss mechanism, prove it with a correlated timeline, make the smallest causal change, and validate the result with a matched A/B run. A capacity number or average GPU-utilization sample is not bubble attribution.
Freeze The Comparison
Record the exact code, container, model revision, precision, hardware, scheduler settings, workload shape, media, prompt, token budget, sampling, cache state, concurrency, and success criteria. Prove semantic correctness before optimizing. Do not compare runtimes when any of these dimensions differ unless that dimension is the declared variable.
Establish The Signal
- Verify actual output-token counts, successful work, fresh-stream coverage, dropped chunks, cancellations, and cleanup. Early EOS or silent failure is not throughput.
- Reproduce at fixed load below the knee, near the highest stable point, and just beyond it when practical. Keep capacity search separate from profiling.
- Record TTFT, inter-token latency, throughput, queue delay, batch size, scheduled tokens, active sequences, preemptions, KV occupancy, GPU memory, and phase durations available in the checked-out version.
- Use scenario-scoped DCGM or
nvidia-smi dmonfor the resource envelope and a short Nsight Systems or repository profiler capture for attribution. Profilers perturb timing; never use a profiled run as the capacity result.
