# Deidentify A Dataset

De-identify selected free-text columns in a local CSV, JSONL, or Parquet dataset with...

- Repository: https://github.com/maziyarpanahi/openmed (5755 stars)
- Category: Data and ML
- License: Apache-2.0
- Last updated: 2026-10-31
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add maziyarpanahi/openmed --skill deidentify-a-dataset -g -a claude-code -y`
Codex: `npx skills add maziyarpanahi/openmed --skill deidentify-a-dataset -g -a codex -y`
Cursor: `npx skills add maziyarpanahi/openmed --skill deidentify-a-dataset -g -a cursor -y`
Gemini CLI: `npx skills add maziyarpanahi/openmed --skill deidentify-a-dataset -g -a gemini-cli -y`
Claude Code plugin marketplace: `/plugin marketplace add maziyarpanahi/openmed`

## Description

De-identify selected free-text columns in a local CSV, JSONL, or Parquet dataset with OpenMed and produce a separate redacted dataset plus a PHI-free aggregate summary. Use when an agent must prepare a clinical dataset for analysis or sharing without overwriting the source or exposing cell values in logs.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# De-identify a dataset

Keep the source local, name the free-text columns explicitly, and write to a
different destination. Never infer columns or print source and redacted cell
values.

## Procedure

1. Confirm that the input is CSV, JSONL/NDJSON, or Parquet.
2. Confirm which columns contain free text. Do not scan or log values to guess.
3. Choose a policy and language. Prefer `strict_no_leak` when recall is the
   governing safety requirement.
4. Write to a new path; never overwrite the input.
5. Inspect only `result.summary`, which contains aggregate counts and rates.
6. Validate recall and residual leakage on representative synthetic or
   approved evaluation fixtures before releasing the output.

## Runnable synthetic example

Install the model runtime first with `python -m pip install "openmed[hf]"`.

```python
import csv
from pathlib import Path

from openmed import redact_dataset

source = Path("synthetic-notes.csv")
destination = Path("synthetic-notes.redacted.csv")

with source.open("w", newline="", encoding="utf-8") as handle:
    writer = csv.DictWriter(handle, fieldnames=["record_id", "note"])
    writer.writeheader()
    writer.writerows(
        [
            {
                "record_id": "SYNTH-001",
                "note": (
                    "Taylor Example called 212-555-0198 about a "
                    "metformin refill."
                ),
            },
            {
                "record_id": "SYNTH-002",
                "note": (
                    "Send the synthetic follow-up to "
                    "demo.patient@example.test."
                ),
            },
        ]
    )

result = redact_dataset(
    source,
    text_columns=["note"],
    output_path=destination,
    policy="strict_no_leak",
    lang="en",
)

print(result.output_path)
print(result.summary.to_dict())  # Aggregate counts only; no cell contents.
```

Use the equivalent CLI for an existing dataset:

The rest: https://github.com/maziyarpanahi/openmed/blob/HEAD/skills/deidentify-a-dataset/SKILL.md