# llama-swap

A model-swapping proxy that runs multiple local AI models and hot-swaps between them on demand, working with any OpenAI or Anthropic API compatible server.

llama-swap sits in front of local inference servers such as llama.cpp and its forks, vLLM, stable-diffusion.cpp, audio.cpp and ComfyUI, letting a user switch which model is loaded without juggling separate processes. It ships as one binary configured by one file, with no other external dependencies.

It implements OpenAI API endpoints for completions, chat, responses, embeddings, model listing, audio speech and transcription, and image generation and editing; Anthropic Messages endpoints; llama-server-specific endpoints such as rerank and infill; and stable-diffusion.cpp's SDAPI. Its own API adds a web UI, a way to list or unload running models, profile activation, and remote log streaming.

It is written in Go and licensed under MIT.

Website: https://github.com/mostlygeek/llama-swap