# Cognee Performance

Use when cognee is slow, expensive, or hitting rate limits, speeding up or throttling...

- Repository: https://github.com/topoteretes/cognee (32985 stars)
- Category: AI and agents
- License: Apache-2.0
- Last updated: 2026-11-01
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add topoteretes/cognee --skill cognee-performance -g -a claude-code -y`
Codex: `npx skills add topoteretes/cognee --skill cognee-performance -g -a codex -y`
Cursor: `npx skills add topoteretes/cognee --skill cognee-performance -g -a cursor -y`
Gemini CLI: `npx skills add topoteretes/cognee --skill cognee-performance -g -a gemini-cli -y`

## Description

Use when cognee is slow, expensive, or hitting rate limits, speeding up or throttling ingestion (remember/cognify batching, chunk size, LLM and embedding rate limits, per-stage models), cutting read latency, estimating cost before ingesting, running work in the background, or planning how to scale a deployment.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Tune cognee's performance

**The LLM is the bottleneck.** Building the graph costs two LLM calls per
chunk (graph extraction and summarization), and almost every other tuning
question comes down to how many of those calls run, how fast, and on which
model. Database and embedding work is small next to that.

## Use it

### 1. Estimate before you ingest

```python
estimate = await cognee.remember("./docs", dry_run=True)
print(estimate)  # per-stage token counts and approximate cost; no LLM calls
```

The estimate covers graph extraction and summarization only, not
`improve()` or embeddings. Not available with
GLiNER or a remote instance.

### 2. Ingestion knobs

All are `remember()` arguments (and `cognify()` ones):

| Knob | Default | What it controls |
|---|---|---|
| `data_per_batch` | 20 | How many documents are processed **at the same time** (a concurrency limit, not a batch). Lower it to reduce load; raise it for many small files. |
| `chunks_per_batch` | 2000 (env `CHUNKS_PER_BATCH`) | How many chunks each extraction/storage call receives. Lower it to spread load and fail smaller. The DLT pipeline defaults to 100. |
| `chunk_size` | min(embedding max tokens, LLM max tokens / 2); about 8191 tokens with default OpenAI models | Max tokens per chunk. Fewer, bigger chunks mean fewer LLM calls; smaller chunks mean finer-grained graphs. |
| `run_in_background` | `False` | Return immediately (`status="running"`); `await result` later. |
| `self_improvement` | `True` | `improve()` after the graph is built. `False` (or `IMPROVE_AUTO_ENABLED=false`) skips that extra work. |

Worst-case concurrent LLM calls are about
`data_per_batch × min(chunks per document, chunks_per_batch) × 2`, and there
is no separate concurrency cap. The rate limiter below is the brake.

Several datasets in one `cognify(datasets=[...])` call run one after
another, not in parallel.

### 3. LLM throttling and models

The rest: https://github.com/topoteretes/cognee/blob/HEAD/.agents/skills/cognee-performance/SKILL.md