Surogate
A C++ and CUDA toolkit that trains and serves large language models on NVIDIA GPUs, combining pretraining, LoRA or QLoRA fine-tuning, GRPO or DPO optimization and HTTP-based serving in one system.
Surogate is a native C++ and CUDA toolkit for training and serving large language models on NVIDIA hardware. It supports pretraining, fine-tuning with LoRA or QLoRA, optimization with GRPO or DPO, and model distillation, then serves the result through HTTP APIs compatible with common client libraries.
The project reports higher training and serving throughput than comparable tools such as Unsloth, vLLM and llama.cpp, using compiled training graphs, fused CUDA kernels, low-precision tensor cores and concurrent serving. It supports model formats including GGUF, NVFP4, BF16 and FP8, and can scale from a single workstation to multiple GPUs and training clusters, or use system RAM to work with models larger than available GPU memory.
It is written in C++ and CUDA.