Gemini Video Understanding Skill

Purpose

This skill enables video understanding workflows using the Google Gemini API, including video summarization, question answering, transcription with optional visual descriptions, timestamp-based queries (MM:SS), scene/timeline detection, video clipping, custom FPS sampling, multi-video comparison, and YouTube URL analysis.

When to Use

• Summarizing a video into key points or chapters
• Answering questions about what happens at specific timestamps (MM:SS)
• Producing a transcript (optionally with visual context) and speaker labels
• Detecting scene changes or building a timeline of events
• Analyzing long videos by clipping to relevant segments or reducing FPS
• Comparing multiple videos (up to 10 videos on Gemini 2.5+)
• Analyzing public YouTube videos directly via URL

Required Libraries

The following Python libraries are required:

from google import genai
from google.genai import types
import os
import time

Input Requirements

• File formats: MP4, MPEG, MOV, AVI, FLV, MPG, WebM, WMV, 3GPP
• Size constraints:
  • Use inline bytes for small files (rule of thumb: <20MB).
  • Use the File API upload flow for larger videos (most real videos).
• YouTube:
  • Video must be public (not private/unlisted) and not age-restricted.
  • Provide a valid YouTube URL.
• Duration / context window (model-dependent):
  • 2M-token models: ~2 hours (default resolution) or ~6 hours (low-res).
  • 1M-token models: ~1 hour (default) or ~3 hours (low-res).
• Timestamps: Use MM:SS (e.g., 01:15) when requesting time-based answers.

Output Schema

All extracted/derived content should be returned as valid JSON conforming to this schema: