MOSS
An open-source, bilingual Chinese-English conversational language model with tool-calling support, developed by Fudan University, including model weights, dialogue training data, and deployment and fine-tuning code.
MOSS is an open-source conversational language model developed by Fudan University that supports bilingual Chinese-English dialogue and tool calling. This repository provides the MOSS-003 model series, along with dialogue training data, and code for deploying and fine-tuning the models locally, including hardware requirements and usage examples.
The base model was pretrained on around 700 billion words of Chinese and English text, then further fine-tuned on roughly 1.1 million turns of multi-turn dialogue data to add instruction-following and multi-turn conversation ability, with a further plugin-augmented variant fine-tuned to also use a search engine, a text-to-image tool, a calculator and an equation solver. Quantized 4-bit and 8-bit versions of both variants are also provided, reducing the graphics memory needed for inference to about 12 or 24 gigabytes respectively.
Related sibling projects from the same research group, referenced from this repository, include a text-to-speech model, a vision-language model and a further multimodal project.