dLLM
A library that unifies training, inference, and evaluation of diffusion language models, aiming to make that pipeline transparent and reproducible.
dLLM is a library that unifies the training and evaluation of diffusion language models, an alternative to autoregressive generation, aiming for transparency and reproducibility across the whole development pipeline. It provides scalable training pipelines built on the Hugging Face Trainer, with support for LoRA, DeepSpeed, and FSDP, and unified evaluation pipelines built on the lm-evaluation-harness that abstract away inference details.
On top of these components, it provides minimal training, inference, and evaluation recipes for open-weight diffusion models such as LLaDA and Dream, plus implementations of training algorithms including masked diffusion, block diffusion, and Edit Flows. Recent additions include reinforcement-learning training for masked diffusion models, accelerated inference and evaluation, and recipes for converting an existing autoregressive model, such as Qwen, LLaMA, or GPT-2, or a BERT encoder, into a diffusion-based chat model.
It is written in Python and released under the Apache License 2.0.