Coconut
The official research implementation for training large language models to reason using a continuous latent space instead of generating explicit chain-of-thought text.
Coconut is Meta's official code release for training large language models to reason in a continuous latent space, an alternative to producing an explicit chain-of-thought in text. Training data is a JSON list of question, answer and step triples, such as the augmented GSM8K dataset that a bundled preprocessing script can download and prepare.
A run's method and settings are controlled through a YAML configuration file, including which of four training modes to use, coconut, chain-of-thought, coconut without thought, or no chain-of-thought, how many continuous thoughts represent each reasoning step, how many training stages to run, and which Hugging Face model to initialize from. Training and evaluation logging go through Weights and Biases. It is written in Python and licensed under MIT.