# Rerun Chunk Processing

Core mechanics of the Rerun Chunk Processing API (rerun.chunk), LazyChunkStream...

- Repository: https://github.com/rerun-io/rerun (11675 stars)
- Category: Data and ML
- License: Apache-2.0
- Last updated: 2026-11-01
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add rerun-io/rerun --skill rerun-chunk-processing -g -a claude-code -y`
Codex: `npx skills add rerun-io/rerun --skill rerun-chunk-processing -g -a codex -y`
Cursor: `npx skills add rerun-io/rerun --skill rerun-chunk-processing -g -a cursor -y`
Gemini CLI: `npx skills add rerun-io/rerun --skill rerun-chunk-processing -g -a gemini-cli -y`

## Description

Core mechanics of the Rerun Chunk Processing API (rerun.chunk), LazyChunkStream pipelines, Chunk, lenses (MutateLens/DeriveLens/Selector), RrdReader, writing optimized RRDs. Read BEFORE writing any ingestion/conversion/preprocessing code (convert an MCAP, build a recording from a dataset, preprocess an .rrd, port an old converter): it mandates reader+lens pipelines and steers away from hand-built chunks, no Chunk.from_columns for data a reader/lens can produce, no per-message rr.log, no manual pa.array assembly. Source-specific knowledge lives in the importer skills (rerun-mcap, rerun-urdf, rerun-parquet, rerun-mp4, rerun-lerobot); read rerun-data-model first to decide what the data should become.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Rerun chunk processing

The pipeline layer between raw data and an RRD: readers produce `Chunk`s,
streams transform them, terminal calls execute. This skill is the generic
mechanics only. Decide the data model first (`rerun-data-model`), then pick the
importer skill for each source:

| Source                                    | Reader                                  | Skill           |
| ----------------------------------------- | --------------------------------------- | --------------- |
| MCAP file (ROS2, protobuf, Foxglove)      | `McapReader(path).stream()`             | `rerun-mcap`    |
| URDF robot model (+ joint states → FK)    | `UrdfTree.from_file_path(...).stream()` | `rerun-urdf`    |
| Parquet table (trajectories, sensor logs) | `ParquetReader(path).stream()`          | `rerun-parquet` |
| mp4 camera video                          | `Mp4Reader(path).stream()`              | `rerun-mp4`     |
| HDF5 file (groups → entities)             | `Hdf5Reader(path).stream()`             | none yet        |
| LeRobot dataset directory                 | built-in importer, then `RrdReader`     | `rerun-lerobot` |
| Existing RRD                              | `RrdReader(path)`                       | here, below     |
| Sidecar files (JSON calib, metadata)      | `Chunk.from_columns` + `from_iter`      | here, below     |

The API is `rerun.chunk`; when
behavior matters, check the installed surface:
`python -c "from rerun.chunk import LazyChunkStream; help(LazyChunkStream)"`.

## Decision rule: where does each component come from?

Default: **a reader produces the chunks; lenses shape them.** Walk this before
writing any conversion code, most "build it by hand" instincts are wrong here:

The rest: https://github.com/rerun-io/rerun/blob/HEAD/skills/rerun-chunk-processing/SKILL.md