# LiteLLM

LiteLLM is a Rust-core, Python-SDK gateway that gives one OpenAI-format interface to more than a hundred LLM providers, with cost tracking, guardrails, load balancing and logging.

It can be used as a Python SDK for direct in-process calls or deployed as a proxy-server AI gateway with virtual keys, spend tracking and an admin dashboard for a team; supported endpoints include chat completions, responses, embeddings, images, audio, batches, rerank and Anthropic-style messages across providers such as OpenAI, Anthropic, Gemini, Bedrock and Azure.

Beyond model calls, it gateways Agent-to-Agent protocol calls to frameworks including LangGraph, Vertex AI Agent Engine, Azure AI Foundry and Bedrock AgentCore, and bridges MCP servers so their tools can be called from any connected LLM, including through IDEs such as Cursor. The README reports 8 millisecond P95 latency at a thousand requests per second, and lists Netflix among its open-source adopters.

Python, license unspecified ("NOASSERTION") in repository metadata.

Website: https://docs.litellm.ai/docs/