Server Inference
Use this skill when the user wants to run or debug MLX-VLM server inference, including...
Install
npx skills add Blaizzy/mlx-vlm --skill server-inference -g -a claude-code -y- Repository
- Blaizzy/mlx-vlm5.7k
- Category
- Testing
- Updated
- 3d ago
- License
- MIT
- Context cost
- Lightabout 872 tokens
- Runs scripts
- No, instructions only
What it does
Use this skill when the user wants to run or debug MLX-VLM server inference, including uv run mlx_vlm.server, /v1/models, /v1/chat/completions, /v1/responses, streaming, OpenAI-compatible clients, health checks, metrics, model unload/reload, adapters, trust-remote-code, and server request/response failures.
Server Inference
Use this workflow for the FastAPI server and API-compatible inference.
First Checks
- Identify the server command, model, port, request endpoint, request body, and expected response.
- Start with health/model-list checks before debugging generation.
- Separate server startup failures from request-handling failures.
- Keep streaming and non-streaming repros separate.
Startup
uv run mlx_vlm.server \
--model <model-or-path> \
--port 8080
Useful startup flags include --adapter-path, --trust-remote-code, --log-level, --enable-thinking, --thinking-budget, --draft-model, --draft-kind, --kv-bits, --kv-quant-scheme, --max-kv-size, and --vision-cache-size.
Minimal Checks
curl http://127.0.0.1:8080/health
curl http://127.0.0.1:8080/v1/models
curl http://127.0.0.1:8080/metrics
Minimal chat request:
curl -s http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "<model-or-path>",
"messages": [{"role": "user", "content": "Say hello."}],
"max_tokens": 32,
"stream": false
}'
Minimal Responses API request:
curl -s http://127.0.0.1:8080/v1/responses \
-H 'Content-Type: application/json' \
-d '{
"model": "<model-or-path>",
"input": "Say hello.",
"max_output_tokens": 32
}'
Endpoint Surface (beyond chat/responses)
The server exposes more than chat, cover or route to these as needed:
