# Model Pruning

Reduce LLM size and accelerate inference using pruning techniques like Wanda and...

- Repository: https://github.com/Orchestra-Research/AI-Research-SKILLs (14176 stars)
- Category: AI and agents
- License: MIT
- Last updated: 2026-10-11
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add Orchestra-Research/AI-Research-SKILLs --skill model-pruning -g -a claude-code -y`
Codex: `npx skills add Orchestra-Research/AI-Research-SKILLs --skill model-pruning -g -a codex -y`
Cursor: `npx skills add Orchestra-Research/AI-Research-SKILLs --skill model-pruning -g -a cursor -y`
Gemini CLI: `npx skills add Orchestra-Research/AI-Research-SKILLs --skill model-pruning -g -a gemini-cli -y`
Claude Code plugin marketplace: `/plugin marketplace add Orchestra-Research/AI-Research-SKILLs`

## Description

Reduce LLM size and accelerate inference using pruning techniques like Wanda and SparseGPT. Use when compressing models without retraining, achieving 50% sparsity with minimal accuracy loss, or enabling faster inference on hardware accelerators. Covers unstructured pruning, structured pruning, N:M sparsity, magnitude pruning, and one-shot methods.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# Model Pruning: Compressing LLMs

## When to Use This Skill

Use Model Pruning when you need to:
- **Reduce model size** by 40-60% with <1% accuracy loss
- **Accelerate inference** using hardware-friendly sparsity (2-4× speedup)
- **Deploy on constrained hardware** (mobile, edge devices)
- **Compress without retraining** using one-shot methods
- **Enable efficient serving** with reduced memory footprint

**Key Techniques**: Wanda (weights × activations), SparseGPT (second-order), structured pruning, N:M sparsity

**Papers**: Wanda ICLR 2024 (arXiv 2306.11695), SparseGPT (arXiv 2301.00774)

## Installation

```bash
# Wanda implementation
git clone https://github.com/locuslab/wanda
cd wanda
pip install -r requirements.txt

# Optional: SparseGPT
git clone https://github.com/IST-DASLab/sparsegpt
cd sparsegpt
pip install -e .

# Dependencies
pip install torch transformers accelerate
```

## Quick Start

### Wanda Pruning (One-Shot, No Retraining)

**Source**: ICLR 2024 (arXiv 2306.11695)

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load model
model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-2-7b-hf",
    torch_dtype=torch.float16,
    device_map="cuda"
)
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-hf")

# Calibration data (small dataset for activation statistics)
calib_data = [
    "The quick brown fox jumps over the lazy dog.",
    "Machine learning is transforming the world.",
    "Artificial intelligence powers modern applications.",
]

# Wanda pruning function
def wanda_prune(model, calib_data, sparsity=0.5):
    """
    Wanda: Prune by weight magnitude × input activation.

    Args:
        sparsity: Fraction of weights to prune (0.5 = 50%)
    """
    # 1. Collect activation statistics
    activations = {}

The rest: https://github.com/Orchestra-Research/AI-Research-SKILLs/blob/HEAD/19-emerging-techniques/model-pruning/SKILL.md