# Minicpm5 Deploy Sglang

Serve MiniCPM5-1B or MiniCPM5-2B via SGLang as an OpenAI-compatible HTTP server with...

- Repository: https://github.com/OpenBMB/MiniCPM (11776 stars)
- Category: Engineering
- License: Apache-2.0
- Last updated: 2026-09-21
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add OpenBMB/MiniCPM --skill minicpm5-deploy-sglang -g -a claude-code -y`
Codex: `npx skills add OpenBMB/MiniCPM --skill minicpm5-deploy-sglang -g -a codex -y`
Cursor: `npx skills add OpenBMB/MiniCPM --skill minicpm5-deploy-sglang -g -a cursor -y`
Gemini CLI: `npx skills add OpenBMB/MiniCPM --skill minicpm5-deploy-sglang -g -a gemini-cli -y`

## Description

Serve MiniCPM5-1B or MiniCPM5-2B via SGLang as an OpenAI-compatible HTTP server with RadixAttention prefix cache and built-in MiniCPM5 tool-call parsing. Use when the user asks for "SGLang", "RadixAttention", "prefix cache", batch evaluation, tool calling, or wants a high-concurrency NVIDIA-GPU server alternative to vLLM.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Deploy MiniCPM5-1B and MiniCPM5-2B with SGLang

OpenAI-compatible server with RadixAttention prefix cache. Best fit for **tool calling**, batched eval pipelines, and high-concurrency serving.

## Required input

| Var | Example | Default |
| --- | --- | --- |
| `MODEL_PATH` | `openbmb/MiniCPM5-2B` or `openbmb/MiniCPM5-1B`| required; |
| `PORT` | `30000` | `30000` |
| `GPU_ID` | `0` | `0` |
| `CTX_LEN` | `131072` (128 K) | `131072` |
| `MEM_FRAC` | `0.85` | `0.85` |
| `TOOL_PARSER` | `minicpm5` | `minicpm5`; use `auto` if you want template detection |

## Steps

### 1. Install (once)

```bash
pip install "sglang[srt]>=0.5.16"          # latest, requires CUDA 13.x driver
# pip install "sglang==0.5.6.post3"        # fallback for CUDA 12.x driver hosts
```

**For tool calling, install from `main`**, the `minicpm5` parser ([PR #25600](https://github.com/sgl-project/sglang/pull/25600), merged 2026-05-22) is not in any pip release yet (`v0.5.12.post1` was branched earlier). Plain chat works on the pip release; only `--tool-call-parser minicpm5` needs `main`:

```bash
pip install "git+https://github.com/sgl-project/sglang.git@main#subdirectory=python"
```

### 2. Recommended runtime env vars

```bash
export VLLM_WORKER_MULTIPROC_METHOD=spawn
export SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1
export SGLANG_DISABLE_CUDNN_CHECK=1
```

### 3. Launch

```bash
CUDA_VISIBLE_DEVICES=${GPU_ID} python -m sglang.launch_server \
    --model-path "${MODEL_PATH}" \
    --served-model-name MiniCPM5-2B \
    --dtype bfloat16 \
    --context-length ${CTX_LEN} \
    --mem-fraction-static ${MEM_FRAC} \
    --tool-call-parser ${TOOL_PARSER} \
    --host 0.0.0.0 \
    --port ${PORT}
```

Wait for `The server is fired up and ready to roll!` .

The rest: https://github.com/OpenBMB/MiniCPM/blob/HEAD/skills/minicpm5-deploy-sglang/SKILL.md