# CLI Inference

Use this skill when the user wants to run or debug MLX-VLM inference from the command...

- Repository: https://github.com/Blaizzy/mlx-vlm (5707 stars)
- Category: Testing
- License: MIT
- Last updated: 2026-11-01
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add Blaizzy/mlx-vlm --skill cli-inference -g -a claude-code -y`
Codex: `npx skills add Blaizzy/mlx-vlm --skill cli-inference -g -a codex -y`
Cursor: `npx skills add Blaizzy/mlx-vlm --skill cli-inference -g -a cursor -y`
Gemini CLI: `npx skills add Blaizzy/mlx-vlm --skill cli-inference -g -a gemini-cli -y`
Claude Code plugin marketplace: `/plugin marketplace add Blaizzy/mlx-vlm`

## Description

Use this skill when the user wants to run or debug MLX-VLM inference from the command line, including uv run mlx_vlm.generate, image/audio/video inputs, local model paths, Hugging Face model IDs, deterministic repro commands, and CLI errors around processors, prompts, model loading, or missing weights.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# CLI Inference

Use this workflow for `uv run mlx_vlm.generate` and related command-line inference tasks.

## First Checks

1. Identify the model ID or local path, modality, prompt, media files, and expected output.
2. Prefer an existing local model path or cached Hugging Face model when reproducing. Do not download a large model unless the user asks.
3. Check model-specific docs first when the family has a README under `mlx_vlm/models/<family>/README.md`.
4. Use `uv run mlx_vlm.generate --help` to verify current flags before giving a final command.

## Command Patterns

Text:

```bash
uv run mlx_vlm.generate \
  --model <model-or-path> \
  --prompt "Write a short answer." \
  --max-tokens 128
```

Image:

```bash
uv run mlx_vlm.generate \
  --model <model-or-path> \
  --image /path/to/image.jpg \
  --prompt "Describe this image." \
  --max-tokens 128
```

Audio or multimodal:

```bash
uv run mlx_vlm.generate \
  --model <model-or-path> \
  --image /path/to/image.jpg \
  --audio /path/to/audio.wav \
  --prompt "Describe what you see and hear." \
  --max-tokens 128
```

## More Flags & Modalities

`uv run mlx_vlm.generate --help` is the source of truth (the CLI lives in the `mlx_vlm/generate/` package). Beyond `--model/--prompt/--image/--audio/--max-tokens`, common flags:

- `--chat`, interactive multi-turn session.
- `--adapter-path`, apply a LoRA adapter.
- `--eos-tokens` and sampling flags, use greedy/low-temp when debugging quality.
- `--trust-remote-code`, needed for some custom processors.
- `--kv-bits` / `--kv-quant-scheme` / `--max-kv-size`, quantized/bounded KV cache.
- `--draft-model` / `--draft-kind`, speculative decoding.
- **Image generation/editing:** `--output-modality image`, `--output <file>`, and the `--diffusion-*` flags drive diffusion image models, not just text-out VLMs.

## Reproducibility Rules

The rest: https://github.com/Blaizzy/mlx-vlm/blob/HEAD/skills/skills/cli-inference/SKILL.md