Building Livekit Agents
Builds voice and chat AI agents with LiveKit Agents on LiveKit Cloud or a self-hosted...
Install
npx skills add fcakyon/claude-codex-settings --skill building-livekit-agents -g -a claude-code -y- Repository
- fcakyon/claude-codex-settings1.2k
- Category
- Design
- Updated
- 3d ago
- License
- MIT
- Context cost
- Mediumabout 2,486 tokens
- Runs scripts
- No, instructions only
What it does
Builds voice and chat AI agents with LiveKit Agents on LiveKit Cloud or a self-hosted server. Use when the user asks to "build a voice agent", "create a LiveKit agent", "add voice AI to my app", "implement handoffs", "structure an agent workflow", "my agent is slow / too chatty", "it says it booked but nothing was saved", "make it confirm before committing", "it keeps re-asking things the caller already said", or is writing code against the LiveKit Agents SDK. Covers architecture: designing for latency, keeping context small, splitting a monolithic agent into handoffs and tasks, and designing for voice. Also covers keeping the model in charge of meaning while code owns state, approvals, and effects. For API specifics use reading-livekit-docs. To check behavior use debugging-livekit-agents and testing-livekit-agents.
Building LiveKit agents
This skill covers how to structure a voice agent. It has no API specifics, because those change;
get them from reading-livekit-docs.
Where the agent runs and which LiveKit the project uses are separate questions. An agent the user self-hosts (on their own servers instead of LiveKit Cloud's agent hosting) still connects to LiveKit Cloud and can still use LiveKit Inference. Inference is a LiveKit Cloud feature, so it's only off the table when the project runs on LiveKit OSS. The architecture advice applies either way; on LiveKit OSS, models come from each provider's own plugin and API keys.
Models: default to LiveKit Inference
Use LiveKit Inference for each model that the agent uses: STT, LLM, TTS, or a realtime model. Many examples, and much of the LiveKit code that you remember, use a provider plugin for each model. Do not copy that pattern. Inference is better for these reasons:
- No extra credentials. Inference uses the LiveKit Cloud credentials the agent already has. A plugin needs its own provider account, plus an API key in every environment the agent runs in.
- Switching models is cheap. Trying another LLM or voice means changing the model name, not adding a dependency and a secret. Expect to switch several times while you tune latency and voice.
- One dependency and one bill, instead of one per provider.
Use a provider plugin only when:
- the project runs on LiveKit OSS
- the model, voice, or provider feature the user needs isn't available through Inference (check the
current model list with
reading-livekit-docs; don't assume from memory) - the user explicitly wants their own provider account, for example for an existing contract or a custom model that isn't provided by Inference
If the code already uses plugins, leave them unless the user asks you to change them. Mention Inference once as an option.
