# Profile Vllm Performance

Profile vLLM and RT-VLM inference to identify and remove GPU-idle gaps, underfilled...

- Repository: https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization (1959 stars)
- Category: Engineering
- License: Apache-2.0
- Last updated: 2026-11-01
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add NVIDIA-AI-Blueprints/video-search-and-summarization --skill profile-vllm-performance -g -a claude-code -y`
Codex: `npx skills add NVIDIA-AI-Blueprints/video-search-and-summarization --skill profile-vllm-performance -g -a codex -y`
Cursor: `npx skills add NVIDIA-AI-Blueprints/video-search-and-summarization --skill profile-vllm-performance -g -a cursor -y`
Gemini CLI: `npx skills add NVIDIA-AI-Blueprints/video-search-and-summarization --skill profile-vllm-performance -g -a gemini-cli -y`

## Description

Profile vLLM and RT-VLM inference to identify and remove GPU-idle gaps, underfilled batches, transfer stalls, serialized multimodal work, scheduler gaps, or KV pressure. Use this skill when GPU utilization is unexpectedly low or bursty, capacity trails another runtime, TTFT or ITL regresses, or multimodal prefill and decode appear serialized. Not for running a first benchmark without a reproducible fixed-load signal.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Profile vLLM Performance

Find the dominant loss mechanism, prove it with a correlated timeline, make the
smallest causal change, and validate the result with a matched A/B run. A capacity
number or average GPU-utilization sample is not bubble attribution.

## Freeze The Comparison

Record the exact code, container, model revision, precision, hardware, scheduler
settings, workload shape, media, prompt, token budget, sampling, cache state,
concurrency, and success criteria. Prove semantic correctness before optimizing.
Do not compare runtimes when any of these dimensions differ unless that dimension is
the declared variable.

## Establish The Signal

1. Verify actual output-token counts, successful work, fresh-stream coverage,
   dropped chunks, cancellations, and cleanup. Early EOS or silent failure is not
   throughput.
2. Reproduce at fixed load below the knee, near the highest stable point, and just
   beyond it when practical. Keep capacity search separate from profiling.
3. Record TTFT, inter-token latency, throughput, queue delay, batch size, scheduled
   tokens, active sequences, preemptions, KV occupancy, GPU memory, and phase
   durations available in the checked-out version.
4. Use scenario-scoped DCGM or `nvidia-smi dmon` for the resource envelope and a
   short Nsight Systems or repository profiler capture for attribution. Profilers
   perturb timing; never use a profiled run as the capacity result.

The rest: https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization/blob/HEAD/skills/benchmarking/profile-vllm-performance/SKILL.md