Deploy MiniCPM5-2B with vLLM-Ascend

Use this skill for the BF16 / FP16 openbmb/MiniCPM5-2B checkpoint on Huawei Ascend NPU. vllm-ascend is installed alongside a matching vLLM release and supplies the Ascend backend.

Required input

Variable	Example	Default
MODEL_PATH	openbmb/MiniCPM5-2B	required
PORT	8000	8000
ASCEND_RT_VISIBLE_DEVICES	0	0
CTX_LEN	32768	32768; the native maximum is 131072
MEM_FRAC	0.85	0.85
TP_SIZE	1	number of visible devices

Accept either the Hugging Face id or a local directory containing the MiniCPM5-2B model files.

Prerequisites

Use a compatible Linux Ascend environment with CANN, PyTorch, and TorchNPU installed. Keep the vLLM and vLLM-Ascend versions matched. The basic serving recipe below follows the tested vllm==0.18.0 / vllm-ascend==0.18.0 pair.

Device visibility is controlled by ASCEND_RT_VISIBLE_DEVICES; CUDA_VISIBLE_DEVICES does not select Ascend devices.

Install

The prebuilt image is the simplest option for a compatible Ascend host. This is a minimal single-device example; adjust device nodes and driver mounts for the host hardware and image documentation.

docker run -it --net=host --shm-size=1g \
    --device /dev/davinci0 --device /dev/davinci_manager \
    --device /dev/devmm_svm --device /dev/hisi_hdc \
    -v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
    -v /usr/local/dcmi:/usr/local/dcmi \
    -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
    quay.io/ascend/vllm-ascend:v0.18.0

For an existing CANN environment:

pip install vllm==0.18.0 vllm-ascend==0.18.0 \
    --extra-index-url https://download.pytorch.org/whl/cpu