Radicalbit AI Gateway
An LLM proxy with guardrails, rate limiting, multi-strategy routing, semantic caching, an MCP proxy and end-to-end request tracing, configured through a UI that turns a plain-English description into YAML.
Radicalbit AI Gateway sits in front of one or more LLM providers and adds guardrails, rate limiting, routing, semantic caching, an MCP proxy and per-request tracing. Instead of hand-writing configuration, a built-in AI assistant generates the YAML for a route from a plain-English description.
Routes can be selected by token count, time of day or budget consumed, with semantic caching layered on top so application code does not need to change to control cost. Guardrails such as PII detection, prompt-injection checks and an LLM-as-judge check run before a request reaches the model, and every call reports its routing decision, cache hit or miss, guardrail results and upstream latency with no sampling.
It runs with Docker and Docker Compose, needs an API key for at least one model provider, and is provider-agnostic; the project is released under the Apache-2.0 license.