# Minicpm5 Deploy Vllm Ascend

Deploy MiniCPM5-2B with vLLM on Huawei Ascend NPU using vLLM-Ascend.

- Repository: https://github.com/OpenBMB/MiniCPM (11776 stars)
- Category: DevOps
- License: Apache-2.0
- Last updated: 2026-09-21
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add OpenBMB/MiniCPM --skill minicpm5-deploy-vllm-ascend -g -a claude-code -y`
Codex: `npx skills add OpenBMB/MiniCPM --skill minicpm5-deploy-vllm-ascend -g -a codex -y`
Cursor: `npx skills add OpenBMB/MiniCPM --skill minicpm5-deploy-vllm-ascend -g -a cursor -y`
Gemini CLI: `npx skills add OpenBMB/MiniCPM --skill minicpm5-deploy-vllm-ascend -g -a gemini-cli -y`

## Description

Deploy MiniCPM5-2B with vLLM on Huawei Ascend NPU using vLLM-Ascend. Use when the user mentions vLLM-Ascend, Ascend NPU, Huawei Ascend, CANN, torch_npu, davinci devices, or wants an OpenAI-compatible MiniCPM5 server on Ascend hardware.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Deploy MiniCPM5-2B with vLLM-Ascend

Use this skill for the BF16 / FP16 `openbmb/MiniCPM5-2B` checkpoint on Huawei Ascend NPU. `vllm-ascend` is installed alongside a matching vLLM release and supplies the Ascend backend.

## Required input

| Variable | Example | Default |
| --- | --- | --- |
| `MODEL_PATH` | `openbmb/MiniCPM5-2B` | required |
| `PORT` | `8000` | `8000` |
| `ASCEND_RT_VISIBLE_DEVICES` | `0` | `0` |
| `CTX_LEN` | `32768` | `32768`; the native maximum is `131072` |
| `MEM_FRAC` | `0.85` | `0.85` |
| `TP_SIZE` | `1` | number of visible devices |

Accept either the Hugging Face id or a local directory containing the MiniCPM5-2B model files.

## Prerequisites

Use a compatible Linux Ascend environment with CANN, PyTorch, and TorchNPU installed. Keep the vLLM and vLLM-Ascend versions matched. The basic serving recipe below follows the tested `vllm==0.18.0` / `vllm-ascend==0.18.0` pair.

Device visibility is controlled by `ASCEND_RT_VISIBLE_DEVICES`; `CUDA_VISIBLE_DEVICES` does not select Ascend devices.

## Install

The prebuilt image is the simplest option for a compatible Ascend host. This is a minimal single-device example; adjust device nodes and driver mounts for the host hardware and image documentation.

```bash
docker run -it --net=host --shm-size=1g \
    --device /dev/davinci0 --device /dev/davinci_manager \
    --device /dev/devmm_svm --device /dev/hisi_hdc \
    -v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
    -v /usr/local/dcmi:/usr/local/dcmi \
    -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
    quay.io/ascend/vllm-ascend:v0.18.0
```

For an existing CANN environment:

```bash
pip install vllm==0.18.0 vllm-ascend==0.18.0 \
    --extra-index-url https://download.pytorch.org/whl/cpu
```

The rest: https://github.com/OpenBMB/MiniCPM/blob/HEAD/skills/minicpm5-deploy-vllm-ascend/SKILL.md