DiffuCoder
Research code accompanying a paper on masked diffusion language models for code generation, including inference examples and a Coupled-GRPO post-training method.
ml-diffucoder is the code release accompanying the paper DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation. It investigates how diffusion language models generate code differently from autoregressive models, introducing an autoregressiveness score to quantify how closely a diffusion model's generation order follows a left-to-right causal pattern, and finds that code tasks induce less of that pattern than math tasks, and that sampling temperature changes the generation order itself, not just which tokens are chosen.
The repository trains DiffuCoder using the adaptation approach from DiffuLLaMA, and introduces Coupled-GRPO, a post-training method that improves the model's performance by computing losses more efficiently across masked token positions. Trained models are published on Hugging Face, and MLX support for Apple Silicon was in progress at the time of writing.