RAGEN
A reinforcement-learning framework for training multi-turn reasoning agents that also diagnoses failure modes such as reasoning collapse during training.
RAGEN is a reinforcement-learning framework for training large language model agents that reason over multiple turns, built around an algorithm called StarPO, for State-Thinking-Actions-Reward Policy Optimization. It gives flexible control over reasoning processes, reward assignment, and prompt-rollout structure, supporting both trajectory-level and turn-wise training.
Alongside training, the project studies why agent reinforcement learning training goes wrong, diagnosing hidden failure modes such as an "Echo Trap" and a broader "reasoning collapse," and its second version adds reward-variance-based rollout filtering and mutual-information proxy metrics that detect template collapse during training. It ships ten built-in Gym-compatible environments, including Sokoban, FrozenLake, WebShop, and Countdown, and supports adding custom environments.
It is written in Python and released under the MIT license, developed by the MLL Lab at Northwestern University.