NeMo AutoModel
A PyTorch-native distributed training library for fine-tuning large language and vision-language models directly from their Hugging Face checkpoints, with recipes for many current open-weight model families.
NeMo AutoModel is a distributed training library for full-parameter and LoRA fine-tuning of large language models and vision-language models, built to work out of the box with checkpoints from the Hugging Face Hub rather than requiring a separate conversion step.
The project regularly adds support for newly released model families; its documentation lists recipes for models such as DeepSeek, Qwen, GLM, Llama, Gemma, Mistral and various mixture-of-experts and hybrid-attention architectures, with example recipes that combine expert, pipeline and context parallelism, packed sequences, and techniques such as speculative decoding drafters.
Each supported model ships with a documented recipe, a model coverage page and example commands for supervised fine-tuning or LoRA on multi-node GPU clusters.