Karaoke Captions Skill

Generate dynamic, TikTok/Shorts-style karaoke captions with word-level highlight sweeps using ASS subtitles and FFmpeg libass. Supports style matching from a reference screenshot.

Scripts, presets, and references live next to this skill. Run the commands from the skill directory (or pass absolute paths).

Requirements

• macOS with Apple Silicon (MLX Whisper)
• Python 3.10+
• FFmpeg with libass (brew install ffmpeg)

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

reference.png --------(match_style_from_image.py)--------> style.json --+
                                                                   +--(gen_ass.py)--> captions.ass --(render.sh)--> output.mp4
input.mp4 ---------------------------------->(extract_words.py)--> words.json --+

---

File Architecture

File	Purpose	Role
input.mp4	Source video	Input
reference.png	Style reference image (optional)	Visual Prompt
words.json	Word-level timestamps [{"word": ..., "start": ..., "end": ...}]	Persistent Raw Data
style.json	Font, size, highlight color, outline, soft shadow, margin, words per line	Style Config
captions.ass	ASS subtitle script with two-layer blurred drop shadow and inline color overrides	Generated Intermediate
output.mp4	Final video with burned-in subtitles	Output Deliverable

---

Standard Workflow

Step 1. Extract Word Timestamps

python scripts/extract_words.py \
  input.mp4 words.json \
  --model mlx-community/whisper-large-v3-turbo

(For Chinese videos, add --language zh).

Step 2. Select Style or Auto-match from Reference Image