# Model Evaluation CLI

Creates and runs eval suites, compares model scores and adds eval runs to CI.

- Repository: https://github.com/cheaper-inference/OmniRoute (83611 stars)
- Category: Testing
- License: MIT
- Last updated: 2026-11-01
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add cheaper-inference/OmniRoute --skill cli-eval -g -a claude-code -y`
Codex: `npx skills add cheaper-inference/OmniRoute --skill cli-eval -g -a codex -y`
Cursor: `npx skills add cheaper-inference/OmniRoute --skill cli-eval -g -a cursor -y`
Gemini CLI: `npx skills add cheaper-inference/OmniRoute --skill cli-eval -g -a gemini-cli -y`

## Description

Create and run evaluation suites, watch live benchmark progress, view scorecards, compare model performance, and integrate eval runs with CI workflows from the CLI.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

<!-- generated by src/lib/agentSkills/generator.ts; manual edits will be overwritten -->

## Overview

Create and run evaluation suites, watch live benchmark progress, view scorecards, compare model performance, and integrate eval runs with CI workflows from the CLI.

## Quick install

```bash
npm install -g omniroute   # or: npx omniroute
omniroute --version
```

## Subcommands

### `eval`

**Example:**

```bash
omniroute eval
```

### `eval suites`

**Example:**

```bash
omniroute eval suites
```

### `eval list`

**Example:**

```bash
omniroute eval list
```

### `eval get <suiteId>`

**Example:**

```bash
omniroute eval get <suiteId>
```

### `eval create`

**Flags:**

- `--file <path>`

**Example:**

```bash
omniroute eval create
```

### `eval run <suiteId>`

**Flags:**

- `-m, --model <id>`
- `--combo <name>`
- `--concurrency <n>`
- `--tag <tag>`
- `--watch`

**Example:**

```bash
omniroute eval run <suiteId>
```

### `eval list`

**Flags:**

- `--suite <id>`
- `--status <s>`
- `--since <ts>`
- `--limit <n>`

**Example:**

```bash
omniroute eval list
```

### `eval get <runId>`

**Example:**

```bash
omniroute eval get <runId>
```

### `eval results <runId>`

**Flags:**

- `--failed`

**Example:**

```bash
omniroute eval results <runId>
```

### `eval cancel <runId>`

**Flags:**

- `--yes`

**Example:**

```bash
omniroute eval cancel <runId>
```

### `eval scorecard <runId>`

**Example:**

```bash
omniroute eval scorecard <runId>
```

### `simulate [prompt]`

**Flags:**

- `--file <path>`
- `-m, --model <id>`
- `--combo <name>`
- `--reasoning-effort <level>`
- `--thinking-budget <n>`
- `--explain`

**Example:**

```bash
omniroute simulate [prompt]
```

<!-- skill:custom-start -->
<!-- Migrated from skills/omniroute-cli-eval/SKILL.md (preserved curated content) -->

# OmniRoute, CLI Evals

The rest: https://github.com/cheaper-inference/OmniRoute/blob/HEAD/skills/cli-eval/SKILL.md