# Mhc Algorithm

Implement mHC (Manifold-Constrained Hyper-Connections) for stabilizing deep network...

- Repository: https://github.com/benchflow-ai/skillsbench (1888 stars)
- Category: Engineering
- License: Apache-2.0
- Last updated: 2026-10-24
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add benchflow-ai/skillsbench --skill mhc-algorithm -g -a claude-code -y`
Codex: `npx skills add benchflow-ai/skillsbench --skill mhc-algorithm -g -a codex -y`
Cursor: `npx skills add benchflow-ai/skillsbench --skill mhc-algorithm -g -a cursor -y`
Gemini CLI: `npx skills add benchflow-ai/skillsbench --skill mhc-algorithm -g -a gemini-cli -y`

## Description

Implement mHC (Manifold-Constrained Hyper-Connections) for stabilizing deep network training. Use when implementing residual connection improvements with doubly stochastic matrices via Sinkhorn-Knopp algorithm. Based on DeepSeek's 2025 paper (arXiv:2512.24880).

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# mHC: Manifold-Constrained Hyper-Connections

## Overview

mHC (Manifold-Constrained Hyper-Connections) stabilizes deep network training by constraining residual mixing matrices to be **doubly stochastic**. It provides:

- **Stable Training**: Lower gradient norm variance via doubly stochastic constraints
- **Multiple Streams**: Hyper-Connections with learnable mixing across residual streams
- **Sinkhorn Projection**: Log-space Sinkhorn-Knopp algorithm for doubly stochastic projection
- **GPT Integration**: Pattern for wrapping attention and MLP layers

Two components:
- **HyperConnections Module**: Core PyTorch module with H_res, H_pre, H_post matrices
- **Sinkhorn-Knopp**: Log-space projection to doubly stochastic manifold

## Quick Reference

| Topic | Reference |
|-------|-----------|
| Core Concepts & Math | [Core Concepts](references/core-concepts.md) |
| Sinkhorn Algorithm | [Sinkhorn-Knopp](references/sinkhorn-knopp.md) |
| HyperConnections Module | [Module Implementation](references/module-implementation.md) |
| GPT Integration | [GPT Integration](references/gpt-integration.md) |
| Common Pitfalls | [Pitfalls](references/pitfalls.md) |

## Installation

```python
# Required packages
pip install torch einops numpy
```

## Minimal Example

```python
import torch
import torch.nn as nn
from einops import rearrange, einsum

def sinkhorn_knopp(logits, num_iters=20, tau=0.05):
    log_alpha = logits / tau
    for _ in range(num_iters):
        log_alpha = log_alpha - torch.logsumexp(log_alpha, dim=-1, keepdim=True)
        log_alpha = log_alpha - torch.logsumexp(log_alpha, dim=-2, keepdim=True)
    return torch.exp(log_alpha)

class HyperConnections(nn.Module):
    def __init__(self, num_streams, dim, branch=None, layer_idx=0):
        super().__init__()
        self.num_streams = num_streams
        self.branch = branch

The rest: https://github.com/benchflow-ai/skillsbench/blob/HEAD/tasks-extra/mhc-layer-impl/environment/skills/mhc-algorithm/SKILL.md