Deploy MiniCPM5-1B and MiniCPM5-2B with vLLM

OpenAI-compatible server for the BF16 / FP16 MiniCPM5-1B or MiniCPM5-2B checkpoint.

Required input

Var	Example	Default
MODEL_PATH	openbmb/MiniCPM5-2B	required; openbmb/MiniCPM5-1B also works
PORT	8000	8000
GPU_ID	0	0
CTX_LEN	131072 (128 K)	131072; lower if VRAM tight
MEM_FRAC	0.85	0.85; lower on shared GPUs

Steps

1. Install (once)

pip install "vllm>=0.21"          # latest (CUDA 13.x driver hosts)
# pip install "vllm==0.10.1.1"    # fallback for CUDA 12.x driver hosts

2. Launch

CUDA_VISIBLE_DEVICES=${GPU_ID} vllm serve "${MODEL_PATH}" \
    --served-model-name MiniCPM5-2B \
    --dtype bfloat16 \
    --max-model-len ${CTX_LEN} \
    --gpu-memory-utilization ${MEM_FRAC} \
    --port ${PORT}

Wait for Application startup complete in the log.

3. Validate

curl http://localhost:${PORT}/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "MiniCPM5-2B",
        "messages": [{"role": "user", "content": "1+1=?"}],
        "temperature": 1.0, "top_p": 0.95, "max_tokens": 64,
        "chat_template_kwargs": {"enable_thinking": true}
    }'

Expected: choices[0].message.content contains a coherent answer.

Sampling defaults

{"temperature": 1.0, "top_p": 0.95, "chat_template_kwargs": {"enable_thinking": true}}    // MiniCPM5-2B Think
{"temperature": 0.9, "top_p": 0.95, "chat_template_kwargs": {"enable_thinking": true}}    // MiniCPM5-1B Think
{"temperature": 0.7, "top_p": 0.95, "chat_template_kwargs": {"enable_thinking": false}}   // MiniCPM5-1B No-think

Common pitfalls

• (free / total) < MEM_FRAC hard error: lower --gpu-memory-utilization (e.g. 0.5 on a shared GPU).
• OOM at startup with 128 K: drop --max-model-len to 32768 or 8192.

Tool calling (plugin)