Ollama Herd
A local AI load balancer that turns a set of Mac and other devices running Ollama into one cluster, auto-discovering nodes over mDNS and routing language-model, image, speech and embedding requests to the best-fit device.
Ollama Herd auto-discovers Ollama nodes on a local network through mDNS and routes incoming requests, covering language models, image generation, speech-to-text and embeddings, to whichever device scores best on thermal state, memory fit, queue depth, latency, affinity, availability and context fit, behind an OpenAI-compatible API. Its authors position it against paying for cloud inference at scale, arguing that idle personal devices such as spare Mac Minis or Studios can absorb that load instead.
A router process runs on one machine and a lightweight node process runs on each device offering Ollama, with no configuration files needed for the common case; a node can also be pointed at a specific router address to skip mDNS discovery. It installs with pip or Homebrew, includes a live dashboard covering fleet overview, trends, per-app analytics and recommendations, and its Homebrew install requires an explicit trust step for the third-party tap and builds every dependency from source.