Video Perception

You have access to video understanding tools via the claude-video-vision MCP server.

Available Tools

• video_analyze, Analyze video structure with ffmpeg filters (scene changes, silence, motion, etc.). Use this BEFORE extracting frames to plan your strategy.
• video_watch, Extract frames + process audio from a video. Supports variable FPS/resolution per segment.
• video_detail, Drill into specific segments. Separates extraction from viewing, extract many frames, view few at a time.
• video_info, Get video metadata without processing.
• video_configure, Change settings (backend, resolution, enable_index, etc.).
• video_setup, Check/install dependencies.

Workflow

IMPORTANT: You MUST follow these steps in order. Do NOT skip step 2.

1. Always start with video_info to get duration, resolution, and audio presence. If the user gives a YouTube URL, pass the URL directly as path. The MCP server downloads it with yt-dlp, prefers YouTube subtitles/auto-captions for transcription, and falls back to the configured audio backend only when captions are missing, empty, or suspiciously incomplete.

2. REQUIRED for videos > 30s: Call video_analyze BEFORE extracting any frames. This is NOT optional, it gives you structural data to make smart extraction decisions. Select filters relevant to the user's question:

   User intent	Filters to select
   "What happens in this video?"	scene_changes, silence, transcription
   "Find the scene transitions"	scene_changes, black_intervals
   "Are there frozen/stuck parts?"	freeze, blur
   "Is this a talking head or action?"	motion
   "When does the music start?"	silence, loudness
   "Analyze the lighting"	exposure
   "Summarize this lecture"	transcription, scene_changes, silence
   General / unclear intent	scene_changes, silence, transcription