# Rl Policy Optimization

Best practices for reinforcement learning policy optimization.

- Repository: https://github.com/aiming-lab/AutoResearchClaw (14798 stars)
- Category: Design
- License: MIT
- Last updated: 2026-08-19
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add aiming-lab/AutoResearchClaw --skill rl-policy-optimization -g -a claude-code -y`
Codex: `npx skills add aiming-lab/AutoResearchClaw --skill rl-policy-optimization -g -a codex -y`
Cursor: `npx skills add aiming-lab/AutoResearchClaw --skill rl-policy-optimization -g -a cursor -y`
Gemini CLI: `npx skills add aiming-lab/AutoResearchClaw --skill rl-policy-optimization -g -a gemini-cli -y`

## Description

Best practices for reinforcement learning policy optimization. Use when working on RL agents, PPO, SAC, or reward design.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

## RL Policy Optimization Best Practice
Algorithm selection:
- Discrete actions: PPO, DQN, A2C
- Continuous actions: SAC, TD3, PPO
- Multi-agent: MAPPO, QMIX
- Offline: CQL, IQL, Decision Transformer

Training recipe:
- PPO: clip=0.2, lr=3e-4, gamma=0.99, GAE lambda=0.95
- SAC: lr=3e-4, tau=0.005, auto-tune alpha
- Use vectorized environments (e.g., gymnasium.vector)
- Normalize observations and rewards
- Log episode return, episode length, value loss, policy entropy

Evaluation:
- Report mean +/- std over 10+ evaluation episodes
- Use deterministic policy for evaluation
- Compare against random policy and simple baselines
- Report sample efficiency (return vs. env steps)

Common pitfalls:
- Reward shaping can introduce bias
- Seed sensitivity is HIGH, use 5+ seeds
- Hyperparameter sensitivity, do a small sweep