PaddleFormers

PaddleFormers is a training library built on Baidu's PaddlePaddle deep-learning framework, aiming to give the PaddlePaddle ecosystem model interfaces and functionality comparable to Hugging Face Transformers, with training support for both large language models and vision-language models. It supports the main distributed training strategies used for large models, including tensor, pipeline and expert parallelism, together with automatic mixed precision and other acceleration techniques, and reports training performance ahead of Megatron-LM on models such as DeepSeek-V4 and GLM-4.5-Air.

It supports training more than 100 mainstream large language and vision-language models, including the DeepSeek-V3 and GLM-4.5 series, the Qwen2 and Qwen3 families including Qwen3-VL, and Baidu's own ERNIE-4.5, ERNIE-4.5-VL and PaddleOCR-VL models. It covers the full path from pretraining through post-training, including supervised fine-tuning, LoRA variants and DPO, stores trained models in the Safetensors format so they can be used directly with tools such as vLLM or SGLang, and supports training for function-calling and reasoning-style models. It is also adapted to run on several domestic Chinese accelerator chips.