# AI Ml Governance

Governs models and AI systems in production, intended use, evaluation, monitoring...

- Repository: https://github.com/cbrock84/headcount (2554 stars)
- Category: DevOps
- License: MIT
- Last updated: 2026-10-18
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add cbrock84/headcount --skill ai-ml-governance -g -a claude-code -y`
Codex: `npx skills add cbrock84/headcount --skill ai-ml-governance -g -a codex -y`
Cursor: `npx skills add cbrock84/headcount --skill ai-ml-governance -g -a cursor -y`
Gemini CLI: `npx skills add cbrock84/headcount --skill ai-ml-governance -g -a gemini-cli -y`
Claude Code plugin marketplace: `/plugin marketplace add cbrock84/headcount`

## Description

Governs models and AI systems in production, intended use, evaluation, monitoring, human oversight, documentation, and the decision to deploy or retire. Use this before deploying a model or AI feature, when defining evaluation criteria, when a model's behavior has drifted, when assessing AI risk or regulatory exposure, or when deciding whether an AI system is fit for a consequential decision.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# AI and ML governance

> Regimes governing automated decision-making differ by jurisdiction and sector and are changing
> quickly. Anything affecting credit, employment, housing, insurance, healthcare, or education
> carries specific legal obligations, involve Legal & Risk and qualified counsel rather than
> treating it as an engineering question.

## Define intended use before evaluating anything

Write down what the system is for, what it is **not** for, who is affected by its output, and what
happens when it is wrong. Most AI failures are use outside intended scope by someone who did not
know the scope existed.

Then decide the consequence tier, because it sets everything after it:

- **Advisory**, a human decides, the model suggests. Lightest oversight.
- **Assistive**, the model acts, a human reviews before effect.
- **Autonomous**, the model acts with effect. Highest bar, and rarely appropriate where a person is
  materially affected.

## Evaluation

A held-out evaluation set that reflects real inputs, including the awkward ones. Built before
deployment and kept stable, or you cannot compare versions.

- **Measure the failure that matters.** Aggregate accuracy hides the errors you care about. A model
  that is 95% accurate and wrong disproportionately on one group is not 95% good.
- **Evaluate by segment**, always. This is where fairness problems and quiet degradation appear.
- **Both error directions.** False positives and false negatives usually have different costs, and
  the threshold should reflect that ratio rather than a default.
- **Establish a baseline.** Compare against the current process, often a simple rule, not against
  zero. Plenty of models fail to beat the heuristic they replaced.

## Monitoring

Models degrade silently: the world moves, inputs drift, and accuracy falls without any error being
raised.

The rest: https://github.com/cbrock84/headcount/blob/HEAD/plugins/data-analytics/skills/ai-ml-governance/SKILL.md