Batch Processing Clinical Text
Run large-scale batch NER, PII extraction, or de-identification over many clinical...
Install
npx skills add maziyarpanahi/openmed --skill batch-processing-clinical-text -g -a claude-code -y- Repository
- maziyarpanahi/openmed5.8k
- Category
- Data and ML
- Updated
- 4d ago
- License
- Apache-2.0
- Context cost
- Mediumabout 2,005 tokens
- Runs scripts
- No, instructions only
What it does
Run large-scale batch NER, PII extraction, or de-identification over many clinical notes on-device with OpenMed, with sharding, checkpointing, resumability, and append-only JSONL output. Use when the user needs to process a corpus or folder of notes, de-identify a dataset, run NER over thousands of documents, build a resumable batch pipeline, or stream results to JSONL without holding everything in memory. Covers process_batch / BatchProcessor / BatchItem / BatchResult, the operation= selector (analyze_text | extract_pii | deidentify), iter_process streaming, the PHI-safe on_progress callback, chunking long documents, and no-PHI logging. Produces a resumable batch runner over an OpenMed model.
Batch processing clinical text
openmed.processing runs OpenMed over many documents efficiently, with progress
tracking, per-item error isolation, and streaming. It runs fully on-device:
the corpus, the model, and the output never leave the host. This skill shows a
resumable runner, sharded, checkpointed, append-only JSONL, that you can
restart without reprocessing.
When to use this skill
For corpora, folders, or datasets, anything beyond a handful of notes. For a
single note, just call openmed.analyze_text / deidentify directly
(extracting-clinical-entities, deidentifying-clinical-text). For an
always-on HTTP service, see serving-openmed-rest-api.
Quick start
from openmed import process_batch
texts = ["Patient has type 2 diabetes.", "No acute distress. BP 120/80."]
result = process_batch(texts, model_name="disease_detection_superclinical")
print(result.summary()) # PHI-safe counts + timing
print(result.successful_items, "/", result.total_items)
for item in result.get_successful_results():
print(item.id, item.result.to_dict()["entities"]) # spans only; avoid raw text in logs
process_batch(...) is a thin wrapper over BatchProcessor. Real signatures
(openmed/processing/batch.py):
