LiveKit Agents
A Python framework for building realtime, multimodal voice AI agents that run on servers, mixing speech-to-text, language models, text-to-speech and telephony into conversational participants.
LiveKit Agents is a framework for building realtime, programmable participants that run on servers as conversational, multimodal voice agents able to see, hear and understand. It offers flexible integrations across speech-to-text, language model and text-to-speech providers and realtime APIs, built-in job scheduling and dispatch to connect end users to agents, and native support for the Model Context Protocol, so tools from MCP servers can be added with one line of code.
It builds on LiveKit's open-source WebRTC client ecosystem, supporting client applications on all major platforms, and integrates with LiveKit's telephony stack so an agent can place or receive phone calls; RPCs and other data APIs let it exchange structured data with connected clients. A semantic turn-detection model, built with a transformer, helps it recognize when a user has actually finished speaking, reducing interruptions, and a built-in test framework supports writing tests with judges to check an agent's behavior.
It is fully open source and installs with pip, alongside plugins for chosen model providers; a separate JavaScript and TypeScript library covers the same use cases.