# Scoring Checks

Add a new deterministic scoring check in src/scoring/checks/ that evaluates config...

- Repository: https://github.com/caliber-ai-org/ai-setup (1310 stars)
- Category: Engineering
- License: MIT
- Last updated: 2026-09-24
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add caliber-ai-org/ai-setup --skill scoring-checks -g -a claude-code -y`
Codex: `npx skills add caliber-ai-org/ai-setup --skill scoring-checks -g -a codex -y`
Cursor: `npx skills add caliber-ai-org/ai-setup --skill scoring-checks -g -a cursor -y`
Gemini CLI: `npx skills add caliber-ai-org/ai-setup --skill scoring-checks -g -a gemini-cli -y`
Claude Code plugin marketplace: `/plugin marketplace add caliber-ai-org/ai-setup`

## Description

Add a new deterministic scoring check in src/scoring/checks/ that evaluates config quality. Follows the Check[] return pattern, uses point constants from src/scoring/constants.ts, and integrates via filterChecksForTarget() in src/scoring/index.ts. Use when user says 'add scoring check', 'new check', 'modify scoring criteria', or works in src/scoring/checks/. Do NOT use for display changes or refactoring scoring logic.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Adding a Scoring Check

Add a new deterministic check that evaluates a single aspect of AI agent config quality. All checks must be filesystem-based with no network calls or LLM inference.

## Critical

- **Check must be deterministic**: Same filesystem state → same result every time. No randomness, no external APIs.
- **Point values come from constants.ts**: Every `earnedPoints` and `maxPoints` must reference `POINTS_*` from `src/scoring/constants.ts`. Do NOT hardcode numbers.
- **Always return `Check[]` array**: Export a function `check<Category>(dir: string): Check[]` where category is one of: `existence`, `quality`, `grounding`, `accuracy`, `freshness`, `bonus`.
- **Every check must have**: `id` (kebab-case, unique), `name`, `category`, `maxPoints`, `earnedPoints`, `passed`, `detail`, and optional `suggestion`/`fix`.
- **Fix object fields**: `action` (string describing what to do), `data` (context for the fix), `instruction` (user-facing guidance).
- **Register in src/scoring/index.ts**: Add the import and spread the result into the `allChecks` array in `computeLocalScore()`.
- **Target filtering**: If the check is platform-specific (Claude-only, Cursor-only, etc.), add its ID to the appropriate `*_ONLY_CHECKS` set in `constants.ts`.

## Instructions

### Step 1: Define point constants in src/scoring/constants.ts

Verify before proceeding: Is your check measurable with a numeric point value?

Add constants below the appropriate category section (existence, quality, grounding, accuracy, freshness, bonus):

```typescript
// In the appropriate CATEGORY section, e.g., Quality checks (25 pts):
export const POINTS_YOUR_CHECK_NAME = 4; // 1-12 pts typical

// If threshold-based, add a companion array:
export const YOUR_THRESHOLD_ARRAY = [
  { minValue: 10, points: 4 },
  { minValue: 5, points: 2 },
] as const;
```

The rest: https://github.com/caliber-ai-org/ai-setup/blob/HEAD/.agents/skills/scoring-checks/SKILL.md