MCore-Bridge: Making Megatron training as simple as Transformers
A library of Megatron-Core model definitions for large and multimodal models, letting a developer train models such as Qwen3-Next, GLM-5, DeepSeek-V4 and MiniMax with Megatron-Core's parallelism techniques as easily as with Transformers.
MCore-Bridge is a large-language-model and multimodal-model definition library built on the Megatron-Core ecosystem by the ModelScope community. It currently supports more than 300 text-only models and more than 200 multimodal models, including large language models such as Qwen3-Next, GLM-5.1, DeepSeek-V4, MiniMax-2.7, Kimi-K2.5 and GPT-OSS, and multimodal models such as Qwen3.5, Qwen3-Omni, Gemma4, GLM4.6-V, InternVL3.5 and Ovis2.5.
It supports full-parameter and LoRA training compatible with the PEFT ecosystem, several Megatron-Core parallelism strategies including tensor, pipeline, sequence, context, expert and virtual pipeline parallelism, multimodal FP8 training, multi-token prediction, padding-free sequences and packing, and task types including causal language modeling, sequence classification, embedding and reranking. It is compatible with a range of hardware, from A10, A100, H100 and B200 GPUs to RTX-series cards and Ascend NPUs, and can load and save LoRA or full-parameter weights compatible with mainstream inference frameworks such as Transformers, vLLM and SGLang.
It is a Python project distributed under the Apache-2.0 license and installs from PyPI.