AI Gateway
A lightweight, open-source gateway that routes requests to more than 1,600 language, vision, audio and image models through one API, with automatic retries, load balancing, guardrails and an MCP gateway.
The AI Gateway is a lightweight, open-source, enterprise-ready gateway for routing requests to more than 1,600 language, vision, audio and image models through a single, consistent API, aiming to let a team connect to a new model provider in a couple of minutes. It handles automatic retries and fallbacks to reduce downtime, load balancing and conditional routing to scale an application across providers, and configurable guardrails to enforce policy on requests and responses.
It also includes an MCP Gateway for managing Model Context Protocol servers with enterprise authentication and observability, and support for multi-modal requests and agentic workflow integrations. It runs locally with a single command using Node.js and npm, with a local console for viewing logs, and can be deployed with Docker or to Cloudflare, Replit and other platforms; client libraries exist for JavaScript, Python and REST, alongside integrations with the OpenAI SDKs, LangChain, LlamaIndex, AutoGen and CrewAI.
A separate enterprise version adds private deployment to AWS, Azure, GCP, OpenShift or Kubernetes, along with organization management, governance and security features.