# Minicpm5 Deploy Transformers

Run MiniCPM5-1B or MiniCPM5-2B with Hugging Face Transformers for one-shot Python...

- Repository: https://github.com/OpenBMB/MiniCPM (11776 stars)
- Category: Engineering
- License: Apache-2.0
- Last updated: 2026-09-21
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add OpenBMB/MiniCPM --skill minicpm5-deploy-transformers -g -a claude-code -y`
Codex: `npx skills add OpenBMB/MiniCPM --skill minicpm5-deploy-transformers -g -a codex -y`
Cursor: `npx skills add OpenBMB/MiniCPM --skill minicpm5-deploy-transformers -g -a cursor -y`
Gemini CLI: `npx skills add OpenBMB/MiniCPM --skill minicpm5-deploy-transformers -g -a gemini-cli -y`

## Description

Run MiniCPM5-1B or MiniCPM5-2B with Hugging Face Transformers for one-shot Python generation on GPU (bfloat16) or CPU (float32). Use when the user wants a quick Python script, no server, no extra deps, or asks for "Transformers", "AutoModelForCausalLM", "model.generate" with MiniCPM5.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Deploy MiniCPM5-1B and MiniCPM5-2B with HF Transformers

One-shot Python generation. No server. Works on a single GPU (bfloat16) or CPU only (fp32).

## Required input

| Var | Example | Default |
| --- | --- | --- |
| `MODEL_PATH` | `openbmb/MiniCPM5-2B` or local dir | required; `openbmb/MiniCPM5-1B` also works |
| `MODE` | `think` or `nothink` (`nothink` is 1B-only) | `think` |

## Steps

### 1. Install (once)

```bash
pip install -U "transformers>=5.6,<6" "torch>=2.11" accelerate     # latest (CUDA 13.x driver hosts)
# pip install -U "transformers==4.57.3" "torch==2.7.1" accelerate  # fallback for CUDA 12.x driver hosts
```

### 2. Run

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = "${MODEL_PATH}"      # ← replace
tok = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    torch_dtype=torch.bfloat16,    # CPU users: torch.float32 + device_map="cpu"
    device_map="auto",
).eval()

messages = [{"role": "user", "content": "用一句话解释什么是 GQA。"}]
inputs = tok.apply_chat_template(
    messages,
    add_generation_prompt=True,
    enable_thinking=True,
    return_tensors="pt",
    return_dict=True,
).to(model.device)

with torch.no_grad():
    out = model.generate(
        **inputs,
        max_new_tokens=1024,
        do_sample=True,
        temperature=1.0,
        top_p=0.95,
    )

prompt_len = inputs["input_ids"].shape[-1]
print(tok.decode(out[0][prompt_len:], skip_special_tokens=True))
```

For CPU only: change `torch_dtype=torch.float32, device_map="cpu"`. Keep `enable_thinking=True` and `temperature=1.0` for MiniCPM5-2B; use `enable_thinking=False` and `temperature=0.7` only for MiniCPM5-1B No-think mode.

## Sampling defaults

| Mode | `enable_thinking` | `temperature` | `top_p` |
| --- | --- | --- | --- |
| MiniCPM5-2B Think | `True` | 1.0 | 0.95 |
| MiniCPM5-1B Think | `True` | 0.9 | 0.95 |
| MiniCPM5-1B No-think | `False` | 0.7 | 0.95 |

## Validate

The rest: https://github.com/OpenBMB/MiniCPM/blob/HEAD/skills/minicpm5-deploy-transformers/SKILL.md