# Batch Processing Clinical Text

Run large-scale batch NER, PII extraction, or de-identification over many clinical...

- Repository: https://github.com/maziyarpanahi/openmed (5755 stars)
- Category: Data and ML
- License: Apache-2.0
- Last updated: 2026-10-31
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add maziyarpanahi/openmed --skill batch-processing-clinical-text -g -a claude-code -y`
Codex: `npx skills add maziyarpanahi/openmed --skill batch-processing-clinical-text -g -a codex -y`
Cursor: `npx skills add maziyarpanahi/openmed --skill batch-processing-clinical-text -g -a cursor -y`
Gemini CLI: `npx skills add maziyarpanahi/openmed --skill batch-processing-clinical-text -g -a gemini-cli -y`
Claude Code plugin marketplace: `/plugin marketplace add maziyarpanahi/openmed`

## Description

Run large-scale batch NER, PII extraction, or de-identification over many clinical notes on-device with OpenMed, with sharding, checkpointing, resumability, and append-only JSONL output. Use when the user needs to process a corpus or folder of notes, de-identify a dataset, run NER over thousands of documents, build a resumable batch pipeline, or stream results to JSONL without holding everything in memory. Covers process_batch / BatchProcessor / BatchItem / BatchResult, the operation= selector (analyze_text | extract_pii | deidentify), iter_process streaming, the PHI-safe on_progress callback, chunking long documents, and no-PHI logging. Produces a resumable batch runner over an OpenMed model.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Batch processing clinical text

`openmed.processing` runs OpenMed over many documents efficiently, with progress
tracking, per-item error isolation, and streaming. It runs **fully on-device**:
the corpus, the model, and the output never leave the host. This skill shows a
**resumable** runner, sharded, checkpointed, append-only JSONL, that you can
restart without reprocessing.

## When to use this skill

For corpora, folders, or datasets, anything beyond a handful of notes. For a
single note, just call `openmed.analyze_text` / `deidentify` directly
(`extracting-clinical-entities`, `deidentifying-clinical-text`). For an
always-on HTTP service, see `serving-openmed-rest-api`.

## Quick start

```python
from openmed import process_batch

texts = ["Patient has type 2 diabetes.", "No acute distress. BP 120/80."]
result = process_batch(texts, model_name="disease_detection_superclinical")

print(result.summary())          # PHI-safe counts + timing
print(result.successful_items, "/", result.total_items)
for item in result.get_successful_results():
    print(item.id, item.result.to_dict()["entities"])   # spans only; avoid raw text in logs
```

`process_batch(...)` is a thin wrapper over `BatchProcessor`. Real signatures
(`openmed/processing/batch.py`):

The rest: https://github.com/maziyarpanahi/openmed/blob/HEAD/skills/batch-processing-clinical-text/SKILL.md