ggrun
A launcher for llama.cpp and ik_llama.cpp that reads a GGUF model's tensor layout against the VRAM, RAM and per-GPU bandwidth actually available, then places large mixture-of-experts models across a mismatched multi-GPU setup without manual flag tuning.
ggrun computes how to split a large mixture-of-experts model's tensors across a machine's GPUs, RAM and per-GPU bandwidth so the split fits and the inference server starts without running out of memory or needing hand-tuned flags. It is not itself an inference engine: it reads the GGUF file and the machine's hardware, builds a launch plan for the selected backend, checks that the plan fits, starts the server, and keeps the generated launch command visible, while passing through any flag it does not recognize directly to llama-server.
The project's stated aims are running mixture-of-experts models that do not fit neatly in VRAM, finding a fast configuration that is stable at the context length and load the user actually wants, and making a local model usable for Claude Code's parallel-agent workflows without cloud inference. It installs through a shell script on Linux and macOS or a PowerShell script on Windows, both written to survive issues that broke an earlier one-line installer, such as a byte-order mark breaking comment parsing and a piped install hanging on an interactive consent prompt.