aarambh-studio
A from-scratch decoder-only language model implementation in pure Rust, covering tokenization, training, quantization, alignment, multimodal input, and an inference server.
aarambh-studio is a decoder-only language model implementation built with Rust and the Candle library, covering the full engineering path from tokenization and model construction through training, inference, quantization, adapter tuning, alignment, evaluation, multimodal input, safety evaluation, and an OpenAI-compatible inference server, without using Python or PyTorch. Its final planned release adds multi-head latent attention, a native audio modality, sparse mixture-of-experts dispatch on CUDA, multi-node data-parallel training, test-time compute scaling techniques such as best-of-N and self-consistency, an RLAIF preference-tuning pipeline, sandboxed tool execution and multi-agent delegation for the trained model, a from-scratch retrieval-augmented generation implementation, and several model-merging methods.
It is explicitly a source and engineering project: no crate is published to crates.io, and no pretrained checkpoint, adapter, tokenizer, or compiled binary is attached, so a user must train a model or supply compatible weights themselves. Model scale ranges from a 25-million-parameter tiny configuration to a 1.3-billion-parameter large one.
It is released under the Apache-2.0 license, and its author states this release completes the project's planned engineering arc with no further roadmap.