# Minicpm5 Finetune Trl

Fine-tune MiniCPM5-1B or MiniCPM5-2B with bare-metal TRL + PEFT, including...

- Repository: https://github.com/OpenBMB/MiniCPM (11776 stars)
- Category: Engineering
- License: Apache-2.0
- Last updated: 2026-09-21
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add OpenBMB/MiniCPM --skill minicpm5-finetune-trl -g -a claude-code -y`
Codex: `npx skills add OpenBMB/MiniCPM --skill minicpm5-finetune-trl -g -a codex -y`
Cursor: `npx skills add OpenBMB/MiniCPM --skill minicpm5-finetune-trl -g -a cursor -y`
Gemini CLI: `npx skills add OpenBMB/MiniCPM --skill minicpm5-finetune-trl -g -a gemini-cli -y`

## Description

Fine-tune MiniCPM5-1B or MiniCPM5-2B with bare-metal TRL + PEFT, including assistant-only loss via a chat-template patch. Use when the user wants minimal Python, no YAML, full control, or asks for "TRL", "SFTTrainer", "PEFT", "LoraConfig", "assistant_only_loss".

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Fine-tune MiniCPM5-1B and MiniCPM5-2B with TRL + PEFT

Bare-metal Python recipe with **assistant-only loss mask**. Minimal abstractions, full control.

## Required input

| Var | Example | Default |
| --- | --- | --- |
| `BASE_MODEL` | `openbmb/MiniCPM5-2B` | required; `openbmb/MiniCPM5-1B` also works |
| `DATA` | path to messages-format jsonl | required |
| `OUTPUT_DIR` | `./runs/minicpm5_trl` | required |

## Steps

### 1. Install (once)

```bash
# latest (CUDA 13.x driver hosts)
pip install "torch>=2.11" "torchvision" \
            "trl>=0.21" "peft>=0.13" "transformers>=5.6,<6" \
            datasets accelerate

# fallback for CUDA 12.x driver hosts:
# pip install "torch==2.7.1" "torchvision==0.22.1" \
#             "trl==0.20.0" "peft==0.11.1" "transformers==4.57.3" \
#             datasets accelerate
```

### 2. Patch the tokenizer with a training-only chat template

Save as `train_lora.py`:

```python
import json, os, torch
from datasets import Dataset
from peft import LoraConfig, get_peft_model
from transformers import AutoModelForCausalLM, AutoTokenizer, set_seed
from trl import SFTConfig, SFTTrainer

BASE = os.environ["BASE_MODEL"]
DATA = os.environ["DATA"]
OUT  = os.environ["OUTPUT_DIR"]

The rest: https://github.com/OpenBMB/MiniCPM/blob/HEAD/skills/minicpm5-finetune-trl/SKILL.md