Relax
Relax is an asynchronous reinforcement-learning framework for post-training large multimodal language models at scale, decoupling training and inference so agents can learn from text, image, video and audio interactions.
Relax is a reinforcement-learning post-training framework for multimodal large language models, built on Ray Serve with a service-oriented architecture, using Megatron-LM as the training backend and SGLang for inference. A data-transfer system called TransferQueue fully decouples training from inference, and every role in the pipeline runs as an independent Ray Serve deployment with its own elastic scheduling and fault recovery.
It supports multi-turn agentic reinforcement learning with loss masking and flexible termination, elastic scaling of inference engines during training, and dynamic GPU placement between actor and rollout roles. Included algorithms cover PPO, GRPO and several variants, along with a built-in generative reward model in an LLM-as-judge mode and pluggable custom rewards.
It is written in Python and released as open source by the Xiaohongshu AI Infra Team.