MemVanta
A memory-first C++20 inference runtime for quantized GGUF language models on CPU, trading throughput for a large cut in peak memory use compared with llama.cpp.
MemVanta is a low-memory C++20 inference runtime for running quantized Llama-family GGUF models on CPUs with limited RAM. It uses mmap-backed model access, a paged key-value cache, Q4 and Q8 kernels, and bounded adaptive prefetching, and is explicit that it is memory-first rather than faster than llama.cpp.
A published benchmark on a 7B model at Q4_0 quantization reports 3.80 GiB of peak resident memory against 7.24 GiB for a pinned llama.cpp build, a 47.5% reduction, at roughly a quarter of llama.cpp's token generation speed on that run. It is distributed as a native Linux x86-64 CLI, including through a PyPI package that is a distribution shim around the C++ binary rather than a separate Python implementation.