# Kiln Prerelease Check

Run the Kiln pre-release smoke test suite plus the standard CI checks (checks.sh)...

- Repository: https://github.com/Kiln-AI/Kiln (5247 stars)
- Category: Testing
- License: see repository
- Last updated: 2026-11-01
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add Kiln-AI/Kiln --skill kiln-prerelease-check -g -a claude-code -y`
Codex: `npx skills add Kiln-AI/Kiln --skill kiln-prerelease-check -g -a codex -y`
Cursor: `npx skills add Kiln-AI/Kiln --skill kiln-prerelease-check -g -a cursor -y`
Gemini CLI: `npx skills add Kiln-AI/Kiln --skill kiln-prerelease-check -g -a gemini-cli -y`

## Description

Run the Kiln pre-release smoke test suite plus the standard CI checks (checks.sh), diagnose every prerelease test that broke (and why), and write a clean readable report with recommended actions. Read-only, it never edits code. Use when the user wants to validate a release candidate, run prerelease tests, or asks for a "prerelease check / smoke / audit".

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Kiln Pre-release Check

End-to-end pre-release verification and **diagnosis**. This skill:

1. Runs the standard CI checks (`checks.sh`, lint / format / typecheck / unit tests).
2. Runs the curated `@pytest.mark.prerelease` smoke set (real, paid API calls).
3. For every test that broke, works out **what** broke and **why**, including whether the cause is a model that's gone deprecated (per `ml_model_list.py`) or that the provider no longer recognizes the slug.
4. Sweeps the prerelease model pins for staleness and newer model versions.
5. Writes a clean, scan-readable report with a clear **recommended action** for each finding.

**This skill is read-only. It does not change any code, tests, whitelists, or prod source, ever.** Its entire output is a report at `.prerelease/<timestamp>/REPORT.md` (this directory is gitignored). When the right action is to bump a model pin or update the whitelist, the report *recommends* it; the user makes the change.

---

## What "prerelease" means here

A prerelease test is a paid test (it makes real API calls) that we **cannot** meaningfully replace with a mock, its whole point is to verify a live third-party API still behaves how we depend on it. Examples: `test_connect_vertex_live`, basic OpenAI / Claude / Gemini calls, embedding API across providers, reranker integration, Fireworks fine-tune deployment listing, etc.

The set is a **curated subset of `@pytest.mark.paid`**. Tests carry both markers, `@pytest.mark.paid` for the cost gate, `@pytest.mark.prerelease` for the curated selection.

Concretely:

- `--runpaid` runs the full paid suite (slow, expensive, intended for one-off use).
- `--runprerelease` runs **only** the prerelease subset (still paid, but a small, focused, releaseable list). It implies `--runpaid` for those tests.

The rest: https://github.com/Kiln-AI/Kiln/blob/HEAD/.agents/skills/kiln-prerelease-check/SKILL.md