# llmman

A command-line tool that launches coding agents such as Claude Code, Codex or OpenCode against a model running locally or with a hosted provider, packaging models as standard OCI images so they move like containers.

llmman starts a local inference server, downloads a tested llama.cpp build for the machine's GPU, loads a model and runs an agent against it with a single command, or points the same agent at a hosted provider such as Baseten or Groq through one configuration.

Models are stored as OCI images, so they can be pulled from Docker Hub or Hugging Face, packaged and pushed to GHCR, Quay, Harbor or a private mirror, and transferred directly between registries without landing in a local store. It serves unmodified GGUF and safetensors files through upstream llama.cpp, vLLM, sglang and mlx-lm, with no proprietary blob format.

Several `llmman serve` daemons can be pooled together so a request runs on whichever machine already has the model loaded or has the most free capacity. It installs via a shell script, Homebrew, Scoop or Cargo, and is written in Rust.

Website: https://llmmanorg.github.io/