Deploy MiniCPM5-1B and MiniCPM5-2B with SGLang

OpenAI-compatible server with RadixAttention prefix cache. Best fit for tool calling, batched eval pipelines, and high-concurrency serving.

Required input

Var	Example	Default
MODEL_PATH	openbmb/MiniCPM5-2B or openbmb/MiniCPM5-1B	required;
PORT	30000	30000
GPU_ID	0	0
CTX_LEN	131072 (128 K)	131072
MEM_FRAC	0.85	0.85
TOOL_PARSER	minicpm5	minicpm5; use auto if you want template detection

Steps

1. Install (once)

pip install "sglang[srt]>=0.5.16"          # latest, requires CUDA 13.x driver
# pip install "sglang==0.5.6.post3"        # fallback for CUDA 12.x driver hosts

For tool calling, install from main, the minicpm5 parser (PR #25600, merged 2026-05-22) is not in any pip release yet (v0.5.12.post1 was branched earlier). Plain chat works on the pip release; only --tool-call-parser minicpm5 needs main:

pip install "git+https://github.com/sgl-project/sglang.git@main#subdirectory=python"

2. Recommended runtime env vars

export VLLM_WORKER_MULTIPROC_METHOD=spawn
export SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1
export SGLANG_DISABLE_CUDNN_CHECK=1

3. Launch

CUDA_VISIBLE_DEVICES=${GPU_ID} python -m sglang.launch_server \
    --model-path "${MODEL_PATH}" \
    --served-model-name MiniCPM5-2B \
    --dtype bfloat16 \
    --context-length ${CTX_LEN} \
    --mem-fraction-static ${MEM_FRAC} \
    --tool-call-parser ${TOOL_PARSER} \
    --host 0.0.0.0 \
    --port ${PORT}

Wait for The server is fired up and ready to roll! .