# LLM Caching

Implement multi-layer LLM caching with exact match, semantic similarity, and...

- Repository: https://github.com/BagelHole/DevOps-Security-Agent-Skills (1187 stars)
- Category: AI and agents
- License: MIT
- Last updated: 2026-05-22
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add BagelHole/DevOps-Security-Agent-Skills --skill llm-caching -g -a claude-code -y`
Codex: `npx skills add BagelHole/DevOps-Security-Agent-Skills --skill llm-caching -g -a codex -y`
Cursor: `npx skills add BagelHole/DevOps-Security-Agent-Skills --skill llm-caching -g -a cursor -y`
Gemini CLI: `npx skills add BagelHole/DevOps-Security-Agent-Skills --skill llm-caching -g -a gemini-cli -y`

## Description

Implement multi-layer LLM caching with exact match, semantic similarity, and provider-side prompt caching. Reduce API costs by 30, 70%, cut latency, and improve throughput using Redis, GPTCache, and provider caching APIs.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# LLM Caching

Cut LLM costs and latency with exact match, semantic, and provider-side caching layers.

## When to Use This Skill

Use this skill when:
- The same or similar queries are asked repeatedly (FAQ bots, support tools)
- LLM API costs are growing and you need immediate savings
- Serving high request volumes where repeated queries cause bottlenecks
- Implementing prompt caching for long system prompts (Anthropic/OpenAI)
- Building offline-capable AI features that need response persistence

## Caching Layers

```
Request → Exact Cache → Semantic Cache → Provider Cache → LLM API
             ↓ hit            ↓ hit             ↓ hit
           instant          ~5ms           50-80% cheaper
```

## Layer 1: Exact Match Cache (Redis)

```python
import hashlib
import json
import redis
from openai import OpenAI

r = redis.Redis(host="localhost", port=6379, decode_responses=True)
client = OpenAI()

def build_cache_key(model: str, messages: list, temperature: float) -> str:
    """Deterministic key from request parameters."""
    payload = json.dumps({
        "model": model,
        "messages": messages,
        "temperature": temperature,
    }, sort_keys=True)
    return f"llm:exact:{hashlib.sha256(payload.encode()).hexdigest()}"

def cached_completion(model: str, messages: list, temperature: float = 0.0,
                      ttl: int = 3600) -> dict:
    key = build_cache_key(model, messages, temperature)

    # Check cache
    if cached := r.get(key):
        return json.loads(cached)

    # Call API
    response = client.chat.completions.create(
        model=model, messages=messages, temperature=temperature
    )
    result = response.model_dump()

    # Cache result (only cache deterministic responses)
    if temperature == 0.0:
        r.setex(key, ttl, json.dumps(result))

    return result
```

## Layer 2: Semantic Cache (GPTCache)

The rest: https://github.com/BagelHole/DevOps-Security-Agent-Skills/blob/HEAD/devops/ai/llm-caching/SKILL.md