FreeToken Kai
An unofficial fork of the FreeToken local LLM inference engine that benchmarks a user's own GPU with a given model and hands back the fastest settings as a ready-to-use launch profile.
On consumer GPUs such as a single RTX 3060 12GB, it runs gpt-oss-120b or a 35B mixture-of-experts model; on two RTX 3060s it runs a 125B mixture-of-experts model, and it supports Turing-generation cards such as the RTX 2060 that upstream does not.
A built-in browser benchmark tool tries serving settings one at a time on the installed hardware and model, measures throughput, and saves the fastest combination as a launch profile that the profile editor then recommends by default going forward.
It installs from source the same way as upstream FreeToken, with CUDA kernels JIT-compiled on first use; it is an independent, unofficial fork not affiliated with or supported by the original FreeToken maintainers, and keeps the same Apache-2.0 license.