# Minicpm5 Finetune Ms Swift

Fine-tune MiniCPM5-1B or MiniCPM5-2B with ms-swift or Megatron-SWIFT.

- Repository: https://github.com/OpenBMB/MiniCPM (11776 stars)
- Category: Mobile
- License: Apache-2.0
- Last updated: 2026-09-21
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add OpenBMB/MiniCPM --skill minicpm5-finetune-ms-swift -g -a claude-code -y`
Codex: `npx skills add OpenBMB/MiniCPM --skill minicpm5-finetune-ms-swift -g -a codex -y`
Cursor: `npx skills add OpenBMB/MiniCPM --skill minicpm5-finetune-ms-swift -g -a cursor -y`
Gemini CLI: `npx skills add OpenBMB/MiniCPM --skill minicpm5-finetune-ms-swift -g -a gemini-cli -y`

## Description

Fine-tune MiniCPM5-1B or MiniCPM5-2B with ms-swift or Megatron-SWIFT. Use when the user mentions "ms-swift", "swift sft", "swift rlhf", or "megatron sft". MiniCPM5-1B uses the PyPI release with `--template minicpm5`; MiniCPM5-2B requires `ms-swift>=4.6.0.dev0` with `--template minicpm5_2b`. Both use `--model_type llama`.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Fine-tune MiniCPM5-1B and MiniCPM5-2B with ms-swift

Select the version and template that match the model:

| Model | ms-swift version | Arguments |
| --- | --- | --- |
| MiniCPM5-1B | PyPI `4.5.3` | `--model_type llama --template minicpm5` |
| MiniCPM5-2B | `>=4.6.0.dev0` | `--model_type llama --template minicpm5_2b` |

Both models require `transformers>=5.6`.

> **ms-swift 4.x renamed `--train_type` to `--tuner_type`.** Use `--tuner_type lora` for LoRA training.

## Required input

| Var | Example | Default |
| --- | --- | --- |
| `BASE_MODEL` | `openbmb/MiniCPM5-2B` | required; `openbmb/MiniCPM5-1B` is also supported |
| `TEMPLATE` | `minicpm5_2b` | use `minicpm5` for MiniCPM5-1B |
| `DATA` | path to messages-format jsonl | required |
| `OUTPUT_DIR` | `./runs/minicpm5_swift` | required |
| `GPU_ID` | `0` | `0` |

Each line of `DATA`: `{"messages": [{"role":"...","content":"..."}, ...]}`.

## Steps

### 1. Install (once)

For MiniCPM5-1B:

```bash
pip install "ms-swift==4.5.3" "transformers>=5.6"
```

For MiniCPM5-2B:

```bash
git clone https://github.com/modelscope/ms-swift.git
cd ms-swift
git checkout 654e24f17b5d9f40ed4b9ee4c56a723320244db5
pip install -e .
pip install "transformers>=5.6,<5.17"
```

For Megatron-SWIFT:

```bash
pip install mcore-bridge==1.6.4
```

### 2. Train (LoRA SFT)

```bash
CUDA_VISIBLE_DEVICES=${GPU_ID} swift sft \
    --model "${BASE_MODEL}" \
    --model_type llama \
    --template "${TEMPLATE}" \
    --tuner_type lora \
    --dataset "${DATA}" \
    --output_dir "${OUTPUT_DIR}" \
    --num_train_epochs 2 \
    --per_device_train_batch_size 4 \
    --gradient_accumulation_steps 4 \
    --learning_rate 2e-4 \
    --lora_rank 16 --lora_alpha 32 --lora_dropout 0.05 \
    --target_modules q_proj k_proj v_proj o_proj gate_proj up_proj down_proj \
    --max_length 4096 \
    --warmup_ratio 0.03 \
    --bf16 true \
    --logging_steps 10 \
    --save_steps 200
```

The rest: https://github.com/OpenBMB/MiniCPM/blob/HEAD/skills/minicpm5-finetune-ms-swift/SKILL.md