# Xberg Batch Extraction

Use when extracting from many files at once with shared config, bounded parallelism...

- Repository: https://github.com/xberg-io/xberg (9506 stars)
- Category: Engineering
- License: MIT
- Last updated: 2026-11-01
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add xberg-io/xberg --skill xberg-batch-extraction -g -a claude-code -y`
Codex: `npx skills add xberg-io/xberg --skill xberg-batch-extraction -g -a codex -y`
Cursor: `npx skills add xberg-io/xberg --skill xberg-batch-extraction -g -a cursor -y`
Gemini CLI: `npx skills add xberg-io/xberg --skill xberg-batch-extraction -g -a gemini-cli -y`
Claude Code plugin marketplace: `/plugin marketplace add xberg-io/xberg`

## Description

Use when extracting from many files at once with shared config, bounded parallelism, per-file overrides, and error recovery. Covers the `batch` command, `--file-configs`, `--max-concurrent`, and output layout.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

<!--
AI-RULEZ :: GENERATED FILE, DO NOT EDIT
Content-Hash: blake3:324433f2b10c7c9cdc1c06b98c1e427c8cb0ca9ccf034d336e28d49706add520
Source-Hash: blake3:4a90223a333b0f963ae8cdec4f38db88b769132c5dd1b7f39a7fe8e84f57f83e
Schema-Version: v1
-->

# Batch extraction

Use this when processing a directory or glob of documents in one pass.
`xberg batch` shares one extraction config across every file, runs
extractions concurrently, and returns a JSON envelope, failures on
individual files do not abort the run.

## Basic usage

```bash
# Glob expands to many paths; results come back in a JSON envelope (default)
xberg batch *.pdf

# Mixed formats, markdown content for LLM ingestion
xberg batch docs/*.docx --content-format markdown

# Recurse with the shell, then extract
xberg batch $(find ./corpus -name '*.pdf')
```

`batch` defaults to `--format json` (vs `--format text` for single
`extract`). The envelope contains `results`, `total_ms`, `per_file_ms` and, when failures
occur, `errors`. Each `results` entry is a full extracted document. One input may produce
multiple documents, so do not pair results with inputs by position. `per_file_ms` is
aligned with inputs; inspect `errors` for per-input failures.

```bash
xberg batch reports/*.pdf \
  | jq '.results[] | {chars: (.content | length), mime: .mime_type}'
```

## Parallelism

`--max-concurrent` caps how many files extract at once. When omitted, the scheduler derives document concurrency from the total thread budget. Lower it on memory-constrained hosts or when OCR/ML
models are active, since each in-flight extraction holds its own buffers.
Layout-heavy batches are further limited (1 concurrent extraction for
all-PDF-layout batches, 2 for mixed layout):

```bash
# Cap at 4 concurrent extractions
xberg batch scans/*.pdf --ocr true --max-concurrent 4
```

`--max-threads` additionally caps *total* internal threads (Rayon, ONNX
intra-op, the batch semaphore) for tightly constrained environments:

The rest: https://github.com/xberg-io/xberg/blob/HEAD/plugin/.cursor-plugin/skills/xberg-batch-extraction/SKILL.md