# Minicpm5 Finetune Llamafactory

Fine-tune MiniCPM5-1B or MiniCPM5-2B with LLaMA-Factory (YAML-driven SFT / DPO / WebUI).

- Repository: https://github.com/OpenBMB/MiniCPM (11776 stars)
- Category: Engineering
- License: Apache-2.0
- Last updated: 2026-09-21
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add OpenBMB/MiniCPM --skill minicpm5-finetune-llamafactory -g -a claude-code -y`
Codex: `npx skills add OpenBMB/MiniCPM --skill minicpm5-finetune-llamafactory -g -a codex -y`
Cursor: `npx skills add OpenBMB/MiniCPM --skill minicpm5-finetune-llamafactory -g -a cursor -y`
Gemini CLI: `npx skills add OpenBMB/MiniCPM --skill minicpm5-finetune-llamafactory -g -a gemini-cli -y`

## Description

Fine-tune MiniCPM5-1B or MiniCPM5-2B with LLaMA-Factory (YAML-driven SFT / DPO / WebUI). Use when the user wants to fine-tune via LLaMA-Factory, llamafactory-cli, mentions YAML configs, WebUI, or asks for the most-documented community framework.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Fine-tune MiniCPM5-1B and MiniCPM5-2B with LLaMA-Factory

YAML-driven SFT / DPO with WebUI. Most-documented community framework.

## Required input

| Var | Example | Default |
| --- | --- | --- |
| `BASE_MODEL` | `openbmb/MiniCPM5-2B` | required; `openbmb/MiniCPM5-1B` also works |
| `DATA_DIR` | dir containing `dataset_info.json` + jsonl | required |
| `DATASET_NAME` | name registered in `dataset_info.json` | required |
| `OUTPUT_DIR` | `./runs/minicpm5_lf` | required |
| `GPU_ID` | `0` | `0` |

## Steps

### 1. Install (once, in its own venv to avoid breaking vLLM)

```bash
python -m venv .venv-lf && source .venv-lf/bin/activate
# `template: minicpm5` landed after v0.9.5 (the latest PyPI release), install from source.
git clone --depth 1 https://github.com/hiyouga/LLaMA-Factory.git
pip install -e LLaMA-Factory
```

> ⚠️ LLaMA-Factory requires `transformers>=4.55.0,<=5.8.0` (excluding 4.57.0 and 5.6.0), which can clash with a serving stack such as vLLM. Always install it into its own venv.

### 2. Register the dataset (sharegpt / messages format)

`${DATA_DIR}/dataset_info.json`:

```json
{
  "${DATASET_NAME}": {
    "file_name": "your_data.jsonl",
    "formatting": "sharegpt",
    "columns": {"messages": "messages"},
    "tags": {
      "role_tag": "role", "content_tag": "content",
      "user_tag": "user", "assistant_tag": "assistant", "system_tag": "system"
    }
  }
}
```

Each line of `your_data.jsonl`:

```json
{"messages": [{"role":"system","content":"..."}, {"role":"user","content":"..."}, {"role":"assistant","content":"..."}]}
```

### 3. Write the training YAML

Save as `${OUTPUT_DIR}/lora_sft.yaml`:

```yaml
### model
model_name_or_path: ${BASE_MODEL}
trust_remote_code: false

### method
stage: sft
do_train: true
finetuning_type: lora
lora_rank: 16
lora_alpha: 32
lora_target: all                      # all linear layers

The rest: https://github.com/OpenBMB/MiniCPM/blob/HEAD/skills/minicpm5-finetune-llamafactory/SKILL.md