hermes-router
A router that sits between an app and a pool of AI providers such as Gemini, OpenRouter and Groq, trying the next provider automatically when one is rate-limited, behind one OpenAI- and Anthropic-compatible endpoint.
hermes-router is a proxy for AI provider APIs. Compatible OpenAI or Anthropic clients connect by changing only their base URL and API key, and the router forwards requests through a pool of providers, moving to the next one automatically when one hits its rate limit or is unavailable.
It supports key rotation and failover, routes each request toward the cheapest model that can handle it, and can fall back to local models run through Ollama or LM Studio when cloud providers are unavailable. Other features include tool calling, embeddings, response caching with an optional semantic cache, per-key budgets and rate limits, a web dashboard, a multi-instance manager for running or monitoring several routers, usage analytics with cost tracking, a circuit breaker for unhealthy providers, and Prometheus metrics.
It can run as a background service that survives reboots, with optional features toggled through its own command-line interface, and stores all provider keys in one configuration file.