Data Scientist: Hybrid-Engine Data Processing

Answer data questions through the cheapest engine and surface that can prove the answer, and decide where the computation should live before touching the data.

Execution surfaces: resident kernel first

A persistent REPL/eval kernel (many harnesses expose one for JavaScript and Python) is the default surface. Reason: each one-shot process pays roughly a second of spawn-plus-import overhead and re-scans the input file, while a resident connection amortizes both, after a one-time load, repeat queries return in milliseconds. Exploration is repeat queries, so this difference dominates the session.

1. JavaScript kernel (Bun): run scripts/ensure-js-deps.sh once; it prints the absolute import path for @duckdb/node-api. Dynamic-import it, connect once, query across cells.
2. Python kernel: the default surface for Python work. duckdb/numpy/matplotlib are typically resident; Polars and pyarrow come from scripts/ensure-py-deps.sh, which installs them once into a user cache keyed to the kernel's interpreter, sys.path.insert the printed directory and import. The interpreter itself is never mutated.
3. uv lane (uv run --with ...): isolation for a heavy or crash-prone one-shot that should not take the kernel down.
4. No kernel (plain-shell harness): the same engines as one-shots, bun -e for DuckDB-js, uv run python -c for the Python stack, batching several questions per process.

Per-surface patterns and pitfalls: read references/execution-surfaces.md before first use.

Engine selection