# Minicpm5 Deploy Litert

Run MiniCPM5-2B or MiniCPM5-1B on-device with Google's LiteRT-LM runtime, the...

- Repository: https://github.com/OpenBMB/MiniCPM (11776 stars)
- Category: Mobile
- License: Apache-2.0
- Last updated: 2026-09-21
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add OpenBMB/MiniCPM --skill minicpm5-deploy-litert -g -a claude-code -y`
Codex: `npx skills add OpenBMB/MiniCPM --skill minicpm5-deploy-litert -g -a codex -y`
Cursor: `npx skills add OpenBMB/MiniCPM --skill minicpm5-deploy-litert -g -a cursor -y`
Gemini CLI: `npx skills add OpenBMB/MiniCPM --skill minicpm5-deploy-litert -g -a gemini-cli -y`

## Description

Run MiniCPM5-2B or MiniCPM5-1B on-device with Google's LiteRT-LM runtime, the litert-lm CLI or its OpenAI-compatible server on a desktop, the Kotlin API or the AI Edge Gallery app on Android, the same .litertlm bundle on CPU or GPU. Use when the user says "LiteRT", "LiteRT-LM", "litertlm", ".litertlm", "Android", "Edge Gallery", "on-device app", or wants one bundle for phones and desktops.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Deploy MiniCPM5-2B / MiniCPM5-1B with LiteRT-LM (Android / iOS / desktop / IoT)

[LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM), Google's on-device runtime built on [LiteRT](https://github.com/google-ai-edge/litert) (formerly TensorFlow Lite). One `.litertlm` bundle, CPU or GPU, through a CLI, an OpenAI-compatible server, Python, Kotlin and Swift. The bundles are hosted in `litert-community`. Human-readable reference: [`docs/deployment/litert.md`](../../docs/deployment/litert.md).

## Required input

| Var | Example | Default |
| --- | --- | --- |
| `LITERT_REPO` | `litert-community/MiniCPM5-2B` or `litert-community/MiniCPM5-1B` | `litert-community/MiniCPM5-2B` |
| `FILE` | `MiniCPM5-2B_int4.litertlm` (1.55 GB, the phone file) or `MiniCPM5-2B_int8.litertlm` (2.60 GB, reasoning that has to finish); 1B: `minicpm_wi4b32_wi8_afp32.litertlm` (CPU) or `minicpm_wi4b32_wi8_afp32_gpu_opt.litertlm` (GPU) | `MiniCPM5-2B_int4.litertlm` |
| `BACKEND` | `cpu` or `gpu` | `cpu` |
| `THINKING` | `true` (reason first) or `false` (direct answer) | `false` |
| `PROMPT` | `1+1=?` | `1+1=?` |

## Steps

### 1. Install (once)

```bash
uv tool install litert-lm
```

### 2A. Run a pre-converted bundle (desktop CLI)

```bash
litert-lm run --from-huggingface-repo="${LITERT_REPO}" "${FILE}" \
    --backend "${BACKEND}" --thinking "${THINKING:-false}" --prompt "${PROMPT}"
```

The first run downloads the file into `~/.litert-lm/cache/huggingface/<repo>/`; later runs skip the download. With thinking on, the reasoning prints between `[thought]` and `[/thought]` and the answer follows; `--thinking-budget 2048` caps the reasoning (4096 for math); `THINKING=true` makes the model reason first. Sampling as OpenBMB recommends: `--top-k 40 --top-p 0.95 --temperature 1.0` (the CLI's default top-k is 1 = greedy, so temperature alone changes nothing).

### 2B. Android

The rest: https://github.com/OpenBMB/MiniCPM/blob/HEAD/skills/minicpm5-deploy-litert/SKILL.md