# Data Pipeline

Data pipeline and ETL automation - extract, transform, load workflows for data...

- Repository: https://github.com/claude-office-skills/skills (556 stars)
- Category: Engineering
- License: MIT
- Last updated: 2026-01-31
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add claude-office-skills/skills --skill data-pipeline -g -a claude-code -y`
Codex: `npx skills add claude-office-skills/skills --skill data-pipeline -g -a codex -y`
Cursor: `npx skills add claude-office-skills/skills --skill data-pipeline -g -a cursor -y`
Gemini CLI: `npx skills add claude-office-skills/skills --skill data-pipeline -g -a gemini-cli -y`

## Description

Data pipeline and ETL automation - extract, transform, load workflows for data integration and analytics

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Data Pipeline

Build data pipelines and ETL workflows for data integration, transformation, and analytics automation. Based on n8n's data workflow templates.

## Overview

This skill covers:
- Data extraction from multiple sources
- Transformation and cleaning
- Loading to destinations
- Scheduling and monitoring
- Error handling and alerts

---

## ETL Patterns

### Basic ETL Flow

```
┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│   EXTRACT   │───▶│  TRANSFORM  │───▶│    LOAD     │
│             │    │             │    │             │
│ • APIs      │    │ • Clean     │    │ • Database  │
│ • Databases │    │ • Map       │    │ • Warehouse │
│ • Files     │    │ • Aggregate │    │ • Files     │
│ • Webhooks  │    │ • Enrich    │    │ • APIs      │
└─────────────┘    └─────────────┘    └─────────────┘
```

### n8n ETL Workflow

```yaml
workflow: "Daily Sales ETL"
schedule: "2am daily"

nodes:
  # EXTRACT
  - name: "Extract from Shopify"
    type: shopify
    action: get_orders
    filter: created_at >= yesterday
    
  - name: "Extract from Stripe"
    type: stripe
    action: get_payments
    filter: created >= yesterday
    
  # TRANSFORM
  - name: "Merge Data"
    type: merge
    mode: combine_by_key
    key: order_id
    
  - name: "Transform"
    type: code
    code: |
      return items.map(item => ({
        date: item.created_at.split('T')[0],
        order_id: item.id,
        customer_email: item.email,
        total: parseFloat(item.total_price),
        currency: item.currency,
        items: item.line_items.length,
        source: item.source_name,
        payment_status: item.payment.status
      }));
      
  # LOAD
  - name: "Load to BigQuery"
    type: google_bigquery
    action: insert_rows
    table: sales_daily
    
  - name: "Update Google Sheets"
    type: google_sheets
    action: append_rows
    spreadsheet: "Daily Sales Report"
```

---

## Data Sources

### Common Extractors

The rest: https://github.com/claude-office-skills/skills/blob/HEAD/data-pipeline/SKILL.md