# Minicpm5 Deploy Vllm

Serve MiniCPM5-1B or MiniCPM5-2B via vLLM as an OpenAI-compatible HTTP server.

- Repository: https://github.com/OpenBMB/MiniCPM (11776 stars)
- Category: Engineering
- License: Apache-2.0
- Last updated: 2026-09-21
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add OpenBMB/MiniCPM --skill minicpm5-deploy-vllm -g -a claude-code -y`
Codex: `npx skills add OpenBMB/MiniCPM --skill minicpm5-deploy-vllm -g -a codex -y`
Cursor: `npx skills add OpenBMB/MiniCPM --skill minicpm5-deploy-vllm -g -a cursor -y`
Gemini CLI: `npx skills add OpenBMB/MiniCPM --skill minicpm5-deploy-vllm -g -a gemini-cli -y`

## Description

Serve MiniCPM5-1B or MiniCPM5-2B via vLLM as an OpenAI-compatible HTTP server. Use when the user wants high-throughput production serving on NVIDIA GPU, asks for "vLLM", "OpenAI server", "REST API for MiniCPM5", or "production deployment".

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Deploy MiniCPM5-1B and MiniCPM5-2B with vLLM

OpenAI-compatible server for the BF16 / FP16 MiniCPM5-1B or MiniCPM5-2B checkpoint.

## Required input

| Var | Example | Default |
| --- | --- | --- |
| `MODEL_PATH` | `openbmb/MiniCPM5-2B` | required; `openbmb/MiniCPM5-1B` also works |
| `PORT` | `8000` | `8000` |
| `GPU_ID` | `0` | `0` |
| `CTX_LEN` | `131072` (128 K) | `131072`; lower if VRAM tight |
| `MEM_FRAC` | `0.85` | `0.85`; lower on shared GPUs |

## Steps

### 1. Install (once)

```bash
pip install "vllm>=0.21"          # latest (CUDA 13.x driver hosts)
# pip install "vllm==0.10.1.1"    # fallback for CUDA 12.x driver hosts
```

### 2. Launch

```bash
CUDA_VISIBLE_DEVICES=${GPU_ID} vllm serve "${MODEL_PATH}" \
    --served-model-name MiniCPM5-2B \
    --dtype bfloat16 \
    --max-model-len ${CTX_LEN} \
    --gpu-memory-utilization ${MEM_FRAC} \
    --port ${PORT}
```

Wait for `Application startup complete` in the log.

### 3. Validate

```bash
curl http://localhost:${PORT}/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "MiniCPM5-2B",
        "messages": [{"role": "user", "content": "1+1=?"}],
        "temperature": 1.0, "top_p": 0.95, "max_tokens": 64,
        "chat_template_kwargs": {"enable_thinking": true}
    }'
```

Expected: `choices[0].message.content` contains a coherent answer.

## Sampling defaults

```json
{"temperature": 1.0, "top_p": 0.95, "chat_template_kwargs": {"enable_thinking": true}}    // MiniCPM5-2B Think
{"temperature": 0.9, "top_p": 0.95, "chat_template_kwargs": {"enable_thinking": true}}    // MiniCPM5-1B Think
{"temperature": 0.7, "top_p": 0.95, "chat_template_kwargs": {"enable_thinking": false}}   // MiniCPM5-1B No-think
```

## Common pitfalls

- **`(free / total) < MEM_FRAC` hard error**: lower `--gpu-memory-utilization` (e.g. 0.5 on a shared GPU).
- **OOM at startup with 128 K**: drop `--max-model-len` to 32768 or 8192.

## Tool calling (plugin)

The rest: https://github.com/OpenBMB/MiniCPM/blob/HEAD/skills/minicpm5-deploy-vllm/SKILL.md