Port Model to MLX-VLM
Ports a new model architecture into MLX-VLM with config, weights and tests.
Install
npx skills add Blaizzy/mlx-vlm --skill add-new-model -g -a claude-code -y- Repository
- Blaizzy/mlx-vlm5.7k
- Category
- Engineering
- Updated
- 3d ago
- License
- MIT
- Context cost
- Mediumabout 1,522 tokens
- Runs scripts
- No, instructions only
What it does
Use this skill when the user wants to add or port a new model architecture to MLX-VLM, mapping a Hugging Face model_type to a new file under mlx_vlm/models, writing the ModelConfig, matching layer/weight names, reusing a similar existing model, adding a test class, and validating the port. Covers vision-language, language, audio, and diffusion model ports.
Add a New Model
Use this workflow to port a new model to MLX-VLM.
Layout Rules
- New model lives in
mlx_vlm/models/<model_type>/, and the main file is named after theconfig.jsonmodel_type(e.g.model_type: "llava"→mlx_vlm/models/llava/llava.py). The loader resolves the arch by importingmlx_vlm.models.<model_type>(seeMODEL_REMAPPINGinmlx_vlm/utils.pyfor aliases). - Split by concern like the existing families:
language.py,vision.py,config.py,processing_*.py. A new kernel/helper goes in its own file in the model dir, not insidelanguage.py. - Start from a similar existing model in
mlx_vlm/models/and adapt, don't write from scratch.
