Tessera
A from-scratch LLM stack that distills a large teacher model into a small student and serves it efficiently, with custom Triton and CUDA kernels, sharded training and paged-KV serving.
Tessera is a small, from-scratch large language model stack built to distill a large teacher model into a small student model and then serve that student efficiently. It touches most of the pieces involved end to end: custom GPU kernels, sharded training, an inference engine, quantization, and a serving front end.
On the training side it implements a decoder transformer with RMSNorm, RoPE, grouped-query attention and SwiGLU, knowledge-distillation losses, and a from-scratch FSDP/ZeRO-3 implementation with sharded checkpoints. Its kernels include a Triton FlashAttention forward pass and raw CUDA implementations of RMSNorm and attention.
For serving, it offers a block-paged KV cache, a continuous-batching scheduler, speculative decoding, and post-training quantization such as int8, AWQ and FP8. A Rust gateway built on tokio and axum handles HTTP requests and calls into the Python inference engine. It runs and is unit-tested on a laptop CPU or Apple MPS, with GPU kernels falling back to a torch reference implementation when no NVIDIA GPU is available.