verl-agent
An extension of the veRL reinforcement-learning library for training large language and vision-language model agents, using a step-independent multi-turn rollout design that supports long-horizon, many-step tasks and a new algorithm called Group-in-Group Policy Optimization.
verl-agent extends veRL specifically for training LLM and VLM agents with reinforcement learning. Rather than concatenating a full interaction history at each step, it uses a step-independent multi-turn rollout mechanism with fully customizable per-step input structures, history management and memory modules, making it scale to long-horizon, multi-turn training where a task can take dozens of steps to finish.
It provides a diverse set of RL algorithms, including its own Group-in-Group Policy Optimization (GiGPO), described in a NeurIPS 2025 paper, and a suite of parallelized agent environments for both visual and text-based tasks. It supports models including Qwen3, Qwen3-VL, Qwen2.5, Qwen2.5-VL and LLaMA3.2, and includes a modular memory manager and support for LoRA and REINFORCE++.