# Gating Deid Leakage

Add a CI gate that fails the build when an OpenMed de-identification model's recall on...

- Repository: https://github.com/maziyarpanahi/openmed (5755 stars)
- Category: Testing
- License: Apache-2.0
- Last updated: 2026-10-31
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add maziyarpanahi/openmed --skill gating-deid-leakage -g -a claude-code -y`
Codex: `npx skills add maziyarpanahi/openmed --skill gating-deid-leakage -g -a codex -y`
Cursor: `npx skills add maziyarpanahi/openmed --skill gating-deid-leakage -g -a cursor -y`
Gemini CLI: `npx skills add maziyarpanahi/openmed --skill gating-deid-leakage -g -a gemini-cli -y`
Claude Code plugin marketplace: `/plugin marketplace add maziyarpanahi/openmed`

## Description

Add a CI gate that fails the build when an OpenMed de-identification model's recall on a held-out PHI set drops below threshold or any critical identifier leaks. Use when the user wants a pytest test or CLI step that exits nonzero on de-id regression, wants to wire OpenMed's leakage-first release gates into GitHub Actions / CI, needs a recall floor plus zero-leakage assertion against a synthetic held-out set, or wants to block merges that weaken de-identification. Trigger on \"CI gate\", \"fail the build\", \"regression test\", \"de-id recall threshold\", \"block the merge\", \"exit nonzero\", or \"leakage check in CI\" for OpenMed.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Gating De-id Leakage in CI

Logs, baselines, and models drift. The only durable defense is a gate that runs
on every change and **fails closed** when de-identification regresses. This skill
operationalizes OpenMed's leakage-first ethos into a CI check: recall must stay
above the floor and **critical leakage must be exactly zero**, or the build goes
red.

## When to use this skill

- You want a pytest test or CLI step that exits nonzero on de-id regression.
- You need to block PRs that drop PHI recall or introduce a leak.
- You want OpenMed's release gates (`ReleaseGate`, G1a, G8) enforced in CI.
- You maintain a synthetic held-out PHI set and want it checked automatically.

For the full gate semantics see `evaluating-with-leakage-gates`; this skill is
about *wiring it into CI so it fails the build*.

## Quick start, a pytest gate

```python
# tests/eval/test_deid_leakage_gate.py
import pytest
from openmed.eval import run_suite, ReleaseGate, RELEASABLE

RECALL_FLOOR = 0.99          # direct-identifier recall floor
HELD_OUT = "eval/heldout/phi_synthetic.json"   # SYNTHETIC, committed

@pytest.fixture(scope="module")
def gate_report():
    report = run_suite(
        HELD_OUT,
        suite="golden",
        model_name="OpenMed/Privacy-PII-Detection",
        device="cpu",
        metadata={"family": "PII", "tier": "base", "policy": "hipaa_safe_harbor"},
    )
    return ReleaseGate(milestone="v1.6", policy="hipaa_safe_harbor").evaluate(report)

def test_no_critical_leakage(gate_report):
    # Hard zero: one leaked SSN/credit-card is a breach, full stop.
    assert gate_report.critical_leakage_count == 0, "critical PHI leaked"

def test_recall_floor(gate_report):
    low = {
        label: r
        for label, r in gate_report.per_label_recall.items()
        if r < RECALL_FLOOR
    }
    assert not low, f"recall below floor: {low}"

The rest: https://github.com/maziyarpanahi/openmed/blob/HEAD/skills/gating-deid-leakage/SKILL.md