Convert & Quantize

Use this workflow for mlx_vlm.convert, turning a Hugging Face checkpoint into MLX format, optionally quantizing it.

First Checks

1. Confirm the source: a Hugging Face repo id or a local path (--hf-path, alias --model).
2. Confirm the model family is supported in mlx_vlm/models/ (a folder named after the config.json model_type). If not, this is a porting task, switch to Skill("mlx-vlm-skills:add-new-model").
3. Verify current flags before finalizing: uv run mlx_vlm.convert --help.
4. Entry point is mlx_vlm.convert. python -m mlx_vlm.convert is deprecated; use uv run mlx_vlm.convert ... or python -m mlx_vlm convert ....

Command Patterns

Plain convert (no quantization), saves to ./mlx_model by default:

uv run mlx_vlm.convert --hf-path <repo-or-path> --mlx-path ./out-mlx

4-bit affine quantization (RTN, the default method):

uv run mlx_vlm.convert --hf-path <repo-or-path> --mlx-path ./out-4bit -q --q-bits 4 --q-group-size 64

Other quant modes (--q-mode sets its own bit/group defaults):

# mxfp4 (group 32, 4 bit), nvfp4 (group 16, 4 bit), mxfp8 (group 32, 8 bit)
uv run mlx_vlm.convert --hf-path <repo-or-path> --mlx-path ./out-mxfp4 -q --q-mode mxfp4

Mixed-bit recipe (per-layer bit allocation, llama.cpp-style):

# recipes: mixed_2_6 mixed_3_4 mixed_3_5 mixed_3_6 mixed_3_8 mixed_4_6 mixed_4_8
uv run mlx_vlm.convert --hf-path <repo-or-path> --mlx-path ./out-mixed -q --quant-predicate mixed_3_6

AWQ (activation-aware, needs a calibration pass):

uv run mlx_vlm.convert --hf-path <repo-or-path> --mlx-path ./out-awq -q --quant-method awq \
  --calibration multimodal --calibration-data /path/to/media   # or --calibration text (default)

dtype cast only / dequantize:

uv run mlx_vlm.convert --hf-path <repo-or-path> --mlx-path ./out-bf16 --dtype bfloat16
uv run mlx_vlm.convert --hf-path <quantized-repo> --mlx-path ./out-fp -d   # dequantize