# SnapLLM

A local multi-model inference engine, built on llama.cpp and stable-diffusion.cpp, that keeps several models resident in memory at once and switches between them without reloading weights.

SnapLLM is a local inference engine built on llama.cpp and stable-diffusion.cpp for running language, vision and image-diffusion models with a desktop UI, CLI and API. Where a typical single-model workflow loads a model, uses it, unloads it, then loads the next one, SnapLLM instead keeps multiple models loaded at once and selects between them without an avoidable reload while both stay resident.

It exposes an OpenAI-compatible API as a drop-in replacement, automatic GPU and CPU layer distribution based on available VRAM, and persisted key-value cache state with indexed lookup. It supports GGUF text models such as Llama, Mistral and DeepSeek, vision models including LLaVA, and diffusion models such as Stable Diffusion and FLUX, and runs a web UI on one local port with its API on another. It is written in C++.

Website: https://github.com/maheshvaikri-code/snapllm