# Halo

An open-source training framework for large language and multimodal models that runs the same codebase from one GPU to multiple nodes, supporting pretraining, supervised fine-tuning, preference optimization and asynchronous multi-turn reinforcement learning.

Halo is an open-source framework for training large language and multimodal models, covering pretraining, supervised fine-tuning, preference optimization and fully asynchronous multi-turn reinforcement learning on the same codebase, whether run on one GPU or across multiple nodes. It trains Hugging Face models directly, so checkpoints stay in standard SafeTensors format and continue loading with from_pretrained, and supporting a new model family typically needs under 140 lines of integration code.

Parallelism, including expert, context, tensor and a combined expert-tensor form, is layered onto the existing model rather than implemented as a separate distributed model, built on PyTorch's FSDP2, DTensor and DeviceMesh with DeepEP handling communication for expert parallelism; every training method shares this same parallelism and checkpointing code. Reinforcement learning runs asynchronously, with multi-turn environment rollouts executing as Ray actors against a vLLM or SGLang server that overlap with training through a prefetch queue, and updated weights pushed to the serving server over NCCL.

The project reports substantially higher training throughput than a standard TRL setup on eight B300 GPUs, at lower peak memory.

Website: https://whitecircle.com/halo