toptop
A terminal system monitor purpose-built for local LLM inference, showing live tokens per second, VRAM-spill and throttle warnings and other GPU metrics nvidia-smi does not expose, alongside a full htop or btop-class system monitor.
toptop is a Rust system monitor for machines running local model inference. Its AI view auto-discovers running inference servers such as vLLM, llama.cpp, Ollama, TGI, TensorRT-LLM and LM Studio, and reports live tokens per second, KV-cache pressure and preemption, compute-versus-bandwidth breakdowns, TTFT and TPOT latency at p50, p95 and p99, and warnings before a model's memory spills from VRAM to system RAM, which the tool notes can cause a 5 to 20x slowdown; results can be exported as JSON or Prometheus metrics for Grafana, or viewed across a cluster with a fleet view.
A diagnose mode samples for a few seconds and prints a plain-text verdict naming the likely bottleneck, designed to be pasted into a bug report or chat thread. Beneath the AI view, toptop is also a general system monitor in the style of htop or btop.
It installs as a single binary with no dependencies, from a shell script, Homebrew, a .deb package, or a Windows build, and includes a --demo mode that works without a GPU.