Parallax
A distributed inference engine that lets you build your own AI cluster across ordinary machines to serve large language models, with pipeline-parallel sharding and GPU or Mac backends.
Parallax is a fully decentralized inference engine developed by Gradient. It lets a user assemble an AI cluster for model inference across a set of distributed nodes despite differences in their configuration and physical location, hosting local LLMs on personal devices with cross-platform support.
Core features include pipeline-parallel model sharding, paged KV-cache management with continuous batching on Mac, and dynamic request scheduling and routing. The backend combines peer-to-peer communication through Lattica, a GPU backend powered by SGLang and vLLM, and a Mac backend powered by MLX LM.
It supports a range of open models, including DeepSeek-V3.2, MiniMax, GLM, and Kimi-K2, and installs from source with a shell script that sets up a virtual environment before starting the serve command.