Kev Finetune
Fine-tune a Kev decision model (open Jev-style System One model) on a user's own...
Install
npx skills add jaredpalmer/kev --skill kev-finetune -g -a claude-code -y- Repository
- jaredpalmer/kev9.8k
- Category
- DevOps
- Updated
- 8d ago
- License
- Apache-2.0
- Context cost
- Mediumabout 2,661 tokens
- Runs scripts
- Yes, review before use
What it does
Fine-tune a Kev decision model (open Jev-style System One model) on a user's own workload and serve it with calibrated probabilities. Interviews the user, finds existing Jev/TypeSafe questions or labelled data in their codebase, generates enough synthetic training data to measure a gain, trains and calibrates on Modal, scores against the released checkpoint, deploys a TypeSafe-compatible endpoint, and tears everything down. Use when someone wants noul/choice/score questions answered on their own domain (routing, triage, moderation, classification, scoring), wants calibrated confidence for automation thresholds, mentions Jev, TypeSafe, System One or Kev, or asks to fine-tune, evaluate, deploy or clean up Kev.
Fine-tune Kev on the user's workload
Jev (TypeSafe's System One model) answers typed questions about a text without generating tokens, but it is a fixed hosted model: on the user's data it is out of distribution and its probabilities cannot be recalibrated. Kev is the open reconstruction; because it can be trained, you can fine-tune it on a few hundred to a few thousand labelled examples of the user's exact questions and fit its temperature on a held-out slice, so the probabilities it serves are calibrated for their data. This skill takes a user from "I have this decision to automate" to a deployed, measured endpoint, then cleans up. Nothing needs a local GPU or a clone of the Kev repo.
Scripts (scripts/) are short, standard-library Python plus one Modal app. If the user's case does not fit them,
read the script and change it; they are meant to be edited, not worked around.
| Script | Purpose |
|---|---|
extract_workload.py | scan a codebase for Jev / TypeSafe / System One calls and labelled data; draft workload.json |
convert_data.py | CSV / JSONL of existing labelled examples -> Kev records (column mapping) |
generate_data.py | workload spec -> balanced labelled records via any OpenAI-compatible model; --dry-run prints the prompt |
plan_size.py | how many records make the fine-tune vs baseline comparison statistically meaningful |
split_data.py | validate records, split by state into train / calibration / development |
kev_modal.py | Modal: validate, train, evaluate, compare, pull, publish, teardown, and the Serve endpoint |
Run modal as uvx modal ... if it is not installed. Commands below run from the skill directory.
Phase 0: interview
Do not generate anything before you can answer these. Ask what you cannot infer; keep it to one round if possible.
