# VoidLLM

A self-hosted, privacy-first LLM proxy and AI gateway sitting between applications and providers such as OpenAI, Anthropic or Ollama, adding access control, virtual API keys, usage tracking and load balancing, while never storing prompt content.

VoidLLM is a self-hosted LLM proxy and AI gateway, distributed as a single Go binary with sub-2 millisecond proxy overhead, that sits between applications and LLM providers such as OpenAI, Anthropic, Azure, Ollama, vLLM or any custom endpoint. It gives an organization access control, API key management, usage tracking, rate limiting and multi-deployment load balancing from one place.

By architecture rather than as a togglable setting, it never stores, logs or persists prompt or response content; only metadata such as who made a request, which model, token counts and latency is written to its SQLite or PostgreSQL database. Applications speak the OpenAI API to VoidLLM, which authenticates the virtual key, applies role-based access control, rate limits and budgets, resolves a model alias to a real provider and model, and routes to it with automatic failover if a deployment goes down.

It targets teams that self-host LLM infrastructure or use managed providers and need one control plane, cannot log prompts or responses for privacy or compliance reasons, or need organization, team, user and key-level role-based access control, budgets and routing.

Website: https://voidllm.ai