HeartMuLa - Open-Source Music Generation

Overview

HeartMuLa is a family of open-source music foundation models (Apache-2.0) that generates music conditioned on lyrics and tags, with multilingual support. Generates full songs from lyrics + tags. Comparable to Suno for open-source. Includes:

• HeartMuLa - Music language model (3B/7B) for generation from lyrics + tags
• HeartCodec - 12.5Hz music codec for high-fidelity audio reconstruction
• HeartTranscriptor - Whisper-based lyrics transcription
• HeartCLAP - Audio-text alignment model

When to Use

• User wants to generate music/songs from text descriptions
• User wants an open-source Suno alternative
• User wants local/offline music generation
• User asks about HeartMuLa, heartlib, or AI music generation

Hardware Requirements

• Minimum: 8GB VRAM with --lazy_load true (loads/unloads models sequentially)
• Recommended: 16GB+ VRAM for comfortable single-GPU usage
• Multi-GPU: Use --mula_device cuda:0 --codec_device cuda:1 to split across GPUs
• 3B model with lazy_load peaks at ~6.2GB VRAM

Installation Steps

1. Clone Repository

cd ~/  # or desired directory
git clone https://github.com/HeartMuLa/heartlib.git
cd heartlib

2. Create Virtual Environment (Python 3.10 required)

uv venv --python 3.10 .venv
. .venv/bin/activate
uv pip install -e .

3. Fix Dependency Compatibility Issues

IMPORTANT: As of Feb 2026, the pinned dependencies have conflicts with newer packages. Apply these fixes:

# Upgrade datasets (old version incompatible with current pyarrow)
uv pip install --upgrade datasets

# Upgrade transformers (needed for huggingface-hub 1.x compatibility)
uv pip install --upgrade transformers

4. Patch Source Code (Required for transformers 5.x)

Patch 1 - RoPE cache fix in src/heartlib/heartmula/modeling_heartmula.py: