# Mamba Architecture

State-space model with O(n) complexity vs Transformers' O(n²).

- Repository: https://github.com/Orchestra-Research/AI-Research-SKILLs (14176 stars)
- Category: Design
- License: MIT
- Last updated: 2026-10-11
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add Orchestra-Research/AI-Research-SKILLs --skill mamba-architecture -g -a claude-code -y`
Codex: `npx skills add Orchestra-Research/AI-Research-SKILLs --skill mamba-architecture -g -a codex -y`
Cursor: `npx skills add Orchestra-Research/AI-Research-SKILLs --skill mamba-architecture -g -a cursor -y`
Gemini CLI: `npx skills add Orchestra-Research/AI-Research-SKILLs --skill mamba-architecture -g -a gemini-cli -y`
Claude Code plugin marketplace: `/plugin marketplace add Orchestra-Research/AI-Research-SKILLs`

## Description

State-space model with O(n) complexity vs Transformers' O(n²). 5× faster inference, million-token sequences, no KV cache. Selective SSM with hardware-aware design. Mamba-1 (d_state=16) and Mamba-2 (d_state=128, multi-head). Models 130M-2.8B on HuggingFace.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Mamba - Selective State Space Models

## Quick start

Mamba is a state-space model architecture achieving O(n) linear complexity for sequence modeling.

**Installation**:
```bash
# Install causal-conv1d (optional, for efficiency)
pip install causal-conv1d>=1.4.0

# Install Mamba
pip install mamba-ssm
# Or both together
pip install mamba-ssm[causal-conv1d]
```

**Prerequisites**: Linux, NVIDIA GPU, PyTorch 1.12+, CUDA 11.6+

**Basic usage** (Mamba block):
```python
import torch
from mamba_ssm import Mamba

batch, length, dim = 2, 64, 16
x = torch.randn(batch, length, dim).to("cuda")

model = Mamba(
    d_model=dim,      # Model dimension
    d_state=16,       # SSM state dimension
    d_conv=4,         # Conv1d kernel size
    expand=2          # Expansion factor
).to("cuda")

y = model(x)  # O(n) complexity!
assert y.shape == x.shape
```

## Common workflows

### Workflow 1: Language model with Mamba-2

**Complete LM with generation**:
```python
from mamba_ssm.models.mixer_seq_simple import MambaLMHeadModel
from mamba_ssm.models.config_mamba import MambaConfig
import torch

# Configure Mamba-2 LM
config = MambaConfig(
    d_model=1024,           # Hidden dimension
    n_layer=24,             # Number of layers
    vocab_size=50277,       # Vocabulary size
    ssm_cfg=dict(
        layer="Mamba2",     # Use Mamba-2
        d_state=128,        # Larger state for Mamba-2
        headdim=64,         # Head dimension
        ngroups=1           # Number of groups
    )
)

model = MambaLMHeadModel(config, device="cuda", dtype=torch.float16)

# Generate text
input_ids = torch.randint(0, 1000, (1, 20), device="cuda", dtype=torch.long)
output = model.generate(
    input_ids=input_ids,
    max_length=100,
    temperature=0.7,
    top_p=0.9
)
```

### Workflow 2: Use pretrained Mamba models

**Load from HuggingFace**:
```python
from transformers import AutoTokenizer
from mamba_ssm.models.mixer_seq_simple import MambaLMHeadModel

The rest: https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/01-model-architecture/mamba/SKILL.md