Mixed Precision
Use FP16/BF16 mixed precision to accelerate training and reduce memory.
Install
npx skills add aiming-lab/AutoResearchClaw --skill mixed-precision -g -a claude-code -y- Repository
- aiming-lab/AutoResearchClaw14.8k
- Category
- Engineering
- Updated
- 3mo ago
- License
- MIT
- Context cost
- Lightabout 90 tokens
- Runs scripts
- No, instructions only
What it does
Use FP16/BF16 mixed precision to accelerate training and reduce memory. Use when optimizing GPU performance.
Mixed Precision Training Best Practice
Use torch.cuda.amp for automatic mixed precision:
- Wrap forward pass in torch.cuda.amp.autocast()
- Use GradScaler for loss scaling
- BF16 preferred over FP16 on Ampere+ GPUs (RTX 3xxx, A100, RTX 4xxx)
- Watch for NaN gradients, reduce learning rate if needed
- Do NOT use amp with custom CUDA kernels unless tested
