Ghostlink
Ghostlink is a self-hosted, open-source distributed inference fabric that discovers other machines on a LAN and splits a language model's weights across their CPU, GPU and NPU resources, serving the result behind an OpenAI-compatible API with no manual configuration.
Rather than requiring a homogeneous, co-located GPU fleet the way vLLM does, or leaving distribution out entirely the way Ollama and LM Studio do, Ghostlink is built specifically to combine whatever mismatched hardware already sits on a home network - a gaming GPU, an old laptop, an NPU-equipped ultrabook, a Mac - into one authenticated, observable inference cluster with zero YAML and no manual RPC flags. A mixture-of-experts model larger than any single machine's RAM can still run by streaming its experts from disk.
The project documents its comparison against other local and clustered inference tools and backs its benchmark claims with a real two-machine LAN run rather than only synthetic numbers, and is explicit that its experimental zero-copy transport components measure transport framing latency only, not real token generation speed.
It is written in Rust and released under the MIT license.