PufferLib

When to Use This Skill

Use this skill when:

• Training RL agents with PPO on any environment (single or multi-agent)
• Creating custom environments using the PufferEnv API
• Optimizing performance for parallel environment simulation (vectorization)
• Integrating existing environments from Gymnasium, PettingZoo, Atari, Procgen, etc.
• Developing policies with CNN, LSTM, or custom architectures
• Scaling RL to millions of steps per second for faster experimentation
• Multi-agent RL with native multi-agent environment support

1. High-Performance Training (PuffeRL)

PuffeRL is PufferLib's optimized PPO+LSTM training algorithm achieving 1M-4M steps/second.

Quick start training:

## Distributed training

torchrun --nproc_per_node=4 train.py
python
import pufferlib
from pufferlib import PuffeRL