Open-dLLM
Open-dLLM is an open diffusion-based large language model for code generation, releasing its full stack: pretraining pipeline and datasets, inference scripts, an evaluation suite, and model checkpoints, rather than just weights and inference code.
Open-dLLM aims to be the most fully open release of a diffusion-based large language model to date, going from raw data through training, checkpoints, evaluation and inference in one repository, unlike other diffusion LLM releases such as LLaDA or Dream that typically only ship inference scripts and weights. It supports representation alignment, a technique from an accompanying paper for adapting autoregressive language models into diffusion language models with a 4x speedup, instead of training a diffusion model from scratch.
Its evaluation suite covers standard code-generation benchmarks and code-infilling tasks, including HumanEval, HumanEval+, MBPP, MBPP+, HumanEval-Infill and SantaCoder-FIM, built on lm-eval-harness plus custom metrics. Weights and checkpoints are published on Hugging Face, and the environment is managed with micromamba.
It is written in Python and licensed under Apache-2.0.