LLM Gateway

LLM Gateway is an open-source API gateway for large language models that sits as middleware between an application and providers such as OpenAI, Anthropic and Google Vertex AI. It routes requests across multiple providers, manages API keys for each one in a single place, tracks token usage and cost across every interaction, and reports performance metrics for comparing models.

Its API is compatible with the OpenAI request format, so an existing integration can point at it with minimal changes, and it can be used as a hosted service or self-hosted with Docker for full control over data and configuration. The self-hosted setup runs Postgres in a managed Docker volume rather than a bind-mounted host directory, to avoid permission issues during database initialization. It is written in TypeScript, and a Next.js dashboard is included alongside its API for viewing usage.