AINode
A self-hosted AI appliance for NVIDIA GPU boxes that installs an orchestrator per machine, serves models through OpenAI- and Anthropic-compatible APIs, and automatically clusters multiple nodes for cross-node tensor-parallel inference and fine-tuning.
AINode turns an NVIDIA GB10 box (DGX Spark, ASUS GX10) or any NVIDIA GPU machine into a local AI platform. One command installs an orchestrator container and a systemd unit that provides a web UI for chat, cluster topology, and downloads, an OpenAI-compatible API and an Anthropic Messages API routed fleet-wide by model ID, and a decision endpoint that returns calibrated probabilities for typed questions.
vLLM engines start automatically, one container per loaded model, using the engine image and flags each model was validated with, so the orchestrator itself carries no vLLM or CUDA dependency. Additional nodes are found automatically over UDP, model storage is shared over NFS so a model downloads once, and nodes can be combined into cross-node tensor-parallel inference over RoCE with NCCL. Scripted LoRA, QLoRA, and full fine-tuning run on a node's own GPU.
The orchestrator image is about 364 MB and installs with a single shell command.