# Nlp Alignment

Best practices for LLM alignment techniques including RLHF, DPO, and instruction tuning.

- Repository: https://github.com/aiming-lab/AutoResearchClaw (14798 stars)
- Category: AI and agents
- License: MIT
- Last updated: 2026-08-19
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add aiming-lab/AutoResearchClaw --skill nlp-alignment -g -a claude-code -y`
Codex: `npx skills add aiming-lab/AutoResearchClaw --skill nlp-alignment -g -a codex -y`
Cursor: `npx skills add aiming-lab/AutoResearchClaw --skill nlp-alignment -g -a cursor -y`
Gemini CLI: `npx skills add aiming-lab/AutoResearchClaw --skill nlp-alignment -g -a gemini-cli -y`

## Description

Best practices for LLM alignment techniques including RLHF, DPO, and instruction tuning. Use when working on alignment or safety.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

## LLM Alignment Best Practice
Methods:
- RLHF: Train reward model → PPO fine-tuning (complex but powerful)
- DPO: Direct preference optimization (simpler, no reward model needed)
- GRPO: Group relative policy optimization
- SFT: Supervised fine-tuning as alignment baseline

Training recipe:
- Start with SFT on high-quality instruction data
- DPO: lr=5e-7, beta=0.1, batch_size=64
- PPO: lr=1e-6, clip=0.2, KL coeff=0.02
- Use reference model for KL penalty
- Evaluate on safety benchmarks (TruthfulQA, BBQ, etc.)

Common pitfalls:
- Reward hacking: model finds shortcuts to high reward
- Mode collapse: model generates repetitive outputs
- Catastrophic forgetting: loses general capabilities