Fine-tune MiniCPM5-1B and MiniCPM5-2B with unsloth

Single-GPU LoRA / QLoRA. Heavy custom kernels for memory savings (~2× reduction at 4-bit).

⚠️ Two install-time pins (read this BEFORE running):

1. transformers==4.57.3, required if vLLM is in the same env (unsloth's vLLM coexistence patch only handles transformers ≤ 4.57.x).
2. torch==2.7.1 + torchvision==0.22.1, pip install unsloth may pull a cu13 torch wheel that fails on cu12.x drivers (cuda.is_available() returns False).

pip install --force-reinstall "torch==2.7.1" "torchvision==0.22.1" "transformers==4.57.3"

Required input

Var	Example	Default
BASE_MODEL	openbmb/MiniCPM5-2B	required; openbmb/MiniCPM5-1B also works
DATA	path to messages-format jsonl	required
OUTPUT_DIR	./runs/minicpm5_unsloth	required
LOAD_IN_4BIT	True (QLoRA, lowest VRAM) / False (LoRA bf16)	False

Steps

1. Install (once), pin transformers to 4.57.3

pip install "unsloth>=2026.5"
pip install --force-reinstall "transformers==4.57.3"

🔑 transformers==4.57.3 is required if vLLM is in the same env. unsloth's vLLM coexistence patch only handles transformers ≤ 4.57.x; if you skip this pin you'll get a dataclass error at import time.

2. Train, save as train_unsloth.py

import json, os, torch
os.environ.setdefault("CUDA_VISIBLE_DEVICES", "0")

from unsloth import FastLanguageModel
from datasets import Dataset
from trl import SFTTrainer, SFTConfig

BASE = os.environ["BASE_MODEL"]
DATA = os.environ["DATA"]
OUT  = os.environ["OUTPUT_DIR"]

# 1. Load with unsloth's fast wrapper
model, tok = FastLanguageModel.from_pretrained(
    model_name=BASE,
    max_seq_length=2048,
    dtype=torch.bfloat16,
    load_in_4bit=os.environ.get("LOAD_IN_4BIT", "False") == "True",
    full_finetuning=False,
)