# Vss Evaluate Caption Accuracy

Measure whether an RT-VLM configuration change altered caption quality, capture paired...

- Repository: https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization (1959 stars)
- Category: AI and agents
- License: Apache-2.0
- Last updated: 2026-11-01
- Runs scripts: yes, review before use

## Install

Claude Code: `npx skills add NVIDIA-AI-Blueprints/video-search-and-summarization --skill vss-evaluate-caption-accuracy -g -a claude-code -y`
Codex: `npx skills add NVIDIA-AI-Blueprints/video-search-and-summarization --skill vss-evaluate-caption-accuracy -g -a codex -y`
Cursor: `npx skills add NVIDIA-AI-Blueprints/video-search-and-summarization --skill vss-evaluate-caption-accuracy -g -a cursor -y`
Gemini CLI: `npx skills add NVIDIA-AI-Blueprints/video-search-and-summarization --skill vss-evaluate-caption-accuracy -g -a gemini-cli -y`

## Description

Measure whether an RT-VLM configuration change altered caption quality, capture paired baseline and candidate captions for a set of videos, score both against a ground truth with an LLM judge, and emit an accuracy and processing-time table. Use when changing frame selection, decode, or model settings and you need evidence there is no accuracy regression.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

## Purpose

Answer one question with evidence: **did this RT-VLM change make captions worse?**

A configuration change that saves processing time is only useful if caption quality
holds. This skill captures captions twice over the same videos, once with the
change (HYP) and once without (REF), scores both against a ground truth using an
LLM judge, and reports accuracy delta alongside time saved.

## When to Use

- Before/after changing RT-VLM frame selection, decode, or model settings, to prove there is no caption-quality regression
- Quantifying accuracy delta and processing-time saved for a candidate config change
- Verifying a change was behavior-neutral via frame-level provenance, not just matching totals

Not for deploying RT-VLM, general LVS/RTVI benchmarking (see `vss-benchmark-video-summarization`), or one-off caption spot checks with no paired baseline.

## Prerequisites

Everything except the judge runs **inside the RT-VLM container**. The container
needs a GPU, the model on disk, and the `nvdsframeselector` DeepStream plugin
(shipped with DeepStream in the RT-VLM image).

The rest: https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization/blob/HEAD/skills/benchmarking/vss-evaluate-caption-accuracy/SKILL.md