Auditing Deid Leakage
Adversarially scan already-de-identified clinical text for residual identifiers and...
Install
npx skills add maziyarpanahi/openmed --skill auditing-deid-leakage -g -a claude-code -y- Repository
- maziyarpanahi/openmed5.8k
- Category
- Data and ML
- Updated
- 4d ago
- License
- Apache-2.0
- Context cost
- Mediumabout 1,628 tokens
- Runs scripts
- No, instructions only
What it does
Adversarially scan already-de-identified clinical text for residual identifiers and emit a leakage report that blocks release on any hit. Use after OpenMed de-identification when the user asks to verify a redaction, prove no PHI/PII leaked, gate a dataset before sharing, or run a second-pass detector. Covers format and checksum detectors (SSN, Luhn for card numbers, MRN/account patterns, emails, phones, dates), entropy heuristics for high-randomness tokens, severity scoring, and a hard block-on-leak rule. This is the verification half of OpenMed's leakage-first ethos. Hand-off: re-run openmed.extract_pii on the de-id output and diff against expectations. License-free, local-first. Pairs after deidentifying-clinical-text.
Auditing de-id leakage
De-identification is verified, not assumed. A model-driven redaction can miss a structured identifier (an SSN typo'd with spaces, an account number in a footer, a date in an odd format), and a single residual identifier defeats the whole release. This skill is the adversarial second pass: scan the output of de-identification for anything that still looks like an identifier, score it, and block release on any leak. It is the verification half of OpenMed's leakage-first ethos, gate on leakage, not on F1.
When to use
- Right after
deidentifying-clinical-text, before the de-identified text leaves a trust boundary (export, share, train, publish). - When the user wants proof that "no PHI leaked," a release gate, or a CI check that fails the build if any identifier survives.
- As a belt-and-suspenders detector independent of the model that produced the redaction, a deterministic checker catches different failures than the NER.
Run this on the de-identified text, not the original. The original is expected to be full of identifiers.
Quick start
Two complementary passes, a deterministic structural scan plus a model second-pass diff:
import re
import openmed
# Synthetic, the de-identified OUTPUT we are auditing for residual leaks.
deid_text = "Patient [NAME] seen on [DATE]. Backup contact 415-555-0184; acct 4111111111111111."
def luhn_ok(digits: str) -> bool:
nums = [int(d) for d in digits]
nums[-2::-2] = [(2 * d - 9 if 2 * d > 9 else 2 * d) for d in nums[-2::-2]]
return sum(nums) % 10 == 0
