Data pipeline authoring

A data pipeline is NOT a flow. A flow is one runnable that orchestrates steps internally. A data pipeline is a set of independent scripts, each deployed on its own, that form a DAG by reading and writing shared storage assets (DuckLake tables, data tables, S3 objects, volumes, resources) and by declaring execution triggers. The pipeline is visualized and edited at /pipeline/<folder>; every node is a normal workspace script that happens to carry pipeline annotations. When the user asks for a "data pipeline" (or to "ingest / transform / materialize" data across steps), build pipeline-annotated scripts, do NOT build a flow.

Default to DuckDB + DuckLake

A pipeline node that produces a table should almost always be a duckdb node that materializes its output into a DuckLake table with -- materialize ducklake://<name>/<table> (in a DuckDB node the annotation uses SQL -- comment syntax; write the body as a bare SELECT and let the runtime do the write). DuckLake is the default lakehouse store for pipelines and is the shape the pipeline editor is built around, so prefer it unless the work specifically calls for something else:

• postgresql / data tables, only for row-level, OLTP-style mutations against an existing Postgres data table (frequent single-row upserts/updates, transactional reads that an app queries live).
• bun / python3, only for work that doesn't map to SQL: calling an external API, wrangling files, arbitrary glue. When such a node produces tabular data for downstream steps, write it straight into DuckLake (or a data table) from the same script, see "Ingesting from Python / TypeScript" below, rather than inventing a parallel store.

Do not spread a pipeline across postgres, S3, and DuckLake when one DuckLake lake would do; a consistent DuckLake lakehouse is the goal.

Ingesting from Python / TypeScript