# Embedding Optimization

Optimizing vector embeddings for RAG systems through model selection, chunking...

- Repository: https://github.com/ancoleman/ai-design-components (526 stars)
- Category: AI and agents
- License: MIT
- Last updated: 2025-12-11
- Runs scripts: yes, review before use

## Install

Claude Code: `npx skills add ancoleman/ai-design-components --skill embedding-optimization -g -a claude-code -y`
Codex: `npx skills add ancoleman/ai-design-components --skill embedding-optimization -g -a codex -y`
Cursor: `npx skills add ancoleman/ai-design-components --skill embedding-optimization -g -a cursor -y`
Gemini CLI: `npx skills add ancoleman/ai-design-components --skill embedding-optimization -g -a gemini-cli -y`
Claude Code plugin marketplace: `/plugin marketplace add ancoleman/ai-design-components`

## Description

Optimizing vector embeddings for RAG systems through model selection, chunking strategies, caching, and performance tuning. Use when building semantic search, RAG pipelines, or document retrieval systems that require cost-effective, high-quality embeddings.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Embedding Optimization

Optimize embedding generation for cost, performance, and quality in RAG and semantic search systems.

## When to Use This Skill

Trigger this skill when:
- Building RAG (Retrieval Augmented Generation) systems
- Implementing semantic search or similarity detection
- Optimizing embedding API costs (reducing by 70-90%)
- Improving document retrieval quality through better chunking
- Processing large document corpora (thousands to millions of documents)
- Selecting between API-based vs. local embedding models

## Model Selection Framework

Choose the optimal embedding model based on requirements:

**Quick Recommendations:**
- **Startup/MVP:** `all-MiniLM-L6-v2` (local, 384 dims, zero API costs)
- **Production:** `text-embedding-3-small` (API, 1,536 dims, balanced quality/cost)
- **High Quality:** `text-embedding-3-large` (API, 3,072 dims, premium)
- **Multilingual:** `multilingual-e5-base` (local, 768 dims) or Cohere `embed-multilingual-v3.0`

For detailed decision frameworks including cost comparisons, quality benchmarks, and data privacy considerations, see `references/model-selection-guide.md`.

**Model Comparison Summary:**

| Model | Type | Dimensions | Cost per 1M tokens | Best For |
|-------|------|-----------|-------------------|----------|
| all-MiniLM-L6-v2 | Local | 384 | $0 (compute only) | High volume, tight budgets |
| BGE-base-en-v1.5 | Local | 768 | $0 (compute only) | Quality + cost balance |
| text-embedding-3-small | API | 1,536 | $0.02 | General purpose production |
| text-embedding-3-large | API | 3,072 | $0.13 | Premium quality requirements |
| embed-multilingual-v3.0 | API | 1,024 | $0.10 | 100+ language support |

## Chunking Strategies

Select chunking strategy based on content type and use case:

The rest: https://github.com/ancoleman/ai-design-components/blob/HEAD/skills/embedding-optimization/SKILL.md