evals-skills

Skills that guide AI coding agents to help you build product-specific AI evals.

/plugin marketplace add ai-evals-course/evals-skills

9 skills

Build a custom browser-based annotation interface tailored to your data for reviewing...

Run error analysis on a dataset.

Audit an LLM eval pipeline and surface problems: missing error analysis, unvalidated...

Entry point for evals.

Guides evaluation of RAG pipeline retrieval and generation quality.

Create diverse synthetic test inputs for LLM pipeline evaluation using dimension-based...

Calibrate an LLM judge against human labels using data splits, TPR/TNR, and bias...

Write code evaluators for known failure modes with objective rules.

Design LLM-as-Judge evaluators for subjective criteria that code-based checks cannot...