# LLM

A Rust library and command-line tool that lets an application call OpenAI, Anthropic, Ollama, DeepSeek, Google, Cohere, Mistral, Hugging Face, ElevenLabs and other LLM backends through one unified, builder-style API.

LLM is a Rust crate (the name previously belonged to an unrelated, now-archived project) that gives a single API surface across many providers: OpenAI, Anthropic, Ollama, DeepSeek, xAI, Phind, Groq, OpenRouter, Cohere, Mistral, Hugging Face and ElevenLabs. Chat and text-completion requests share two traits, ChatProvider and CompletionProvider, and a builder pattern configures model, temperature, max tokens and timeouts.

Beyond basic completions it supports multi-step chains across different backends, prompt templates with variables, request validation, retry with exponential backoff, scoring-based evaluation across providers in parallel, function and tool calling, structured JSON output, vision input, speech-to-text and text-to-speech, and a sliding-window conversation memory. It can also serve any configured backend as a REST API in OpenAI's own format.

Website: https://github.com/graniet/llm