Minicpm5 Deploy Litert
Run MiniCPM5-2B or MiniCPM5-1B on-device with Google's LiteRT-LM runtime, the...
Install
npx skills add OpenBMB/MiniCPM --skill minicpm5-deploy-litert -g -a claude-code -y- Repository
- OpenBMB/MiniCPM11.8k
- Category
- Mobile
- Updated
- 1mo ago
- License
- Apache-2.0
- Context cost
- Lightabout 1,192 tokens
- Runs scripts
- No, instructions only
What it does
Run MiniCPM5-2B or MiniCPM5-1B on-device with Google's LiteRT-LM runtime, the litert-lm CLI or its OpenAI-compatible server on a desktop, the Kotlin API or the AI Edge Gallery app on Android, the same .litertlm bundle on CPU or GPU. Use when the user says "LiteRT", "LiteRT-LM", "litertlm", ".litertlm", "Android", "Edge Gallery", "on-device app", or wants one bundle for phones and desktops.
Deploy MiniCPM5-2B / MiniCPM5-1B with LiteRT-LM (Android / iOS / desktop / IoT)
LiteRT-LM, Google's on-device runtime built on LiteRT (formerly TensorFlow Lite). One .litertlm bundle, CPU or GPU, through a CLI, an OpenAI-compatible server, Python, Kotlin and Swift. The bundles are hosted in litert-community. Human-readable reference: docs/deployment/litert.md.
Required input
| Var | Example | Default |
|---|---|---|
LITERT_REPO | litert-community/MiniCPM5-2B or litert-community/MiniCPM5-1B | litert-community/MiniCPM5-2B |
FILE | MiniCPM5-2B_int4.litertlm (1.55 GB, the phone file) or MiniCPM5-2B_int8.litertlm (2.60 GB, reasoning that has to finish); 1B: minicpm_wi4b32_wi8_afp32.litertlm (CPU) or minicpm_wi4b32_wi8_afp32_gpu_opt.litertlm (GPU) | MiniCPM5-2B_int4.litertlm |
BACKEND | cpu or gpu | cpu |
THINKING | true (reason first) or false (direct answer) | false |
PROMPT | 1+1=? | 1+1=? |
Steps
1. Install (once)
uv tool install litert-lm
2A. Run a pre-converted bundle (desktop CLI)
litert-lm run --from-huggingface-repo="${LITERT_REPO}" "${FILE}" \
--backend "${BACKEND}" --thinking "${THINKING:-false}" --prompt "${PROMPT}"
The first run downloads the file into ~/.litert-lm/cache/huggingface/<repo>/; later runs skip the download. With thinking on, the reasoning prints between [thought] and [/thought] and the answer follows; --thinking-budget 2048 caps the reasoning (4096 for math); THINKING=true makes the model reason first. Sampling as OpenBMB recommends: --top-k 40 --top-p 0.95 --temperature 1.0 (the CLI's default top-k is 1 = greedy, so temperature alone changes nothing).
