# Omh Data Pipelines

[omh] Data pipeline work -- an ETL or streaming job, a backfill or replay, duplicate...

- Repository: https://github.com/rlaope/oh-my-hermes (5004 stars)
- Category: Engineering
- License: MIT
- Last updated: 2026-11-01
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add rlaope/oh-my-hermes --skill omh-data-pipelines -g -a claude-code -y`
Codex: `npx skills add rlaope/oh-my-hermes --skill omh-data-pipelines -g -a codex -y`
Cursor: `npx skills add rlaope/oh-my-hermes --skill omh-data-pipelines -g -a cursor -y`
Gemini CLI: `npx skills add rlaope/oh-my-hermes --skill omh-data-pipelines -g -a gemini-cli -y`

## Description

[omh] Data pipeline work -- an ETL or streaming job, a backfill or replay, duplicate events, a schema change downstream, a lineage question, a data-quality regression: make every rerun idempotent, bound every replay, and gate each load on observed checks. Use when the user says: data-pipelines, data pipeline, data pipelines, etl, elt, etl pipeline, etl job, etl backfill.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Data Pipelines

This is an OMH `data-pipelines` workflow skill, projected for Agent Skills hosts (Claude Code, Codex, Cursor, opencode, OpenClaw, pi).

## Why This Exists

`data-pipelines` exists because pipeline work had no owner: `backend` owns a service's schema migration, `data-analysis` analyzes data it is handed, and `relational-db` owns a database's locks and indexes, while a backfill that duplicated events or a schema change with unknown readers reached memory and event lanes with no idempotency contract or replay bound at all.

## First Steps

- Ask what makes a row unique at the sink before planning any rerun.
- Bound the window and the targets before ordering any replay or backfill step.

## Do Not Use When

- The ask is a service's own database migration, API, or queue design; use `omh-backend`.
- The ask is analyzing, charting, or summarizing a dataset that was handed over; use `omh-data-analysis`.
- The ask is a slow query, an index, or DDL locking a live table; use `omh-relational-db`.
- The ask is remembering or syncing what the assistant knows about the user.

## Examples

Good example:

- Prompt: our airflow etl backfill is producing duplicate events
- Expected behavior: Find the sink's unique key, name the append that duplicated rows, write the idempotency contract (event-id dedupe or partition overwrite), then bound the backfill window and gate it on key uniqueness and row count against the prior window.
- Why: Rerunning an appending backfill doubles the duplicates it was meant to fix.

Bad example:

- Prompt: just delete the duplicates and rerun the whole history
- Expected behavior: Refuse the unbounded rerun: fix the write to be idempotent first, then backfill a bounded window behind a quality gate.
- Why: Deleting duplicates without fixing the write guarantees the next rerun duplicates again.

## Completion Checklist

The rest: https://github.com/rlaope/oh-my-hermes/blob/HEAD/agent-skills/omh-data-pipelines/SKILL.md