LLM Inspector
A command-line tool that inspects live LLM inference processes and breaks down GPU memory use into weights, KV cache, workspace and activations, then projects how much memory different quantization strategies would save.
LLM Inspector inspects a running inference process and shows exactly how its GPU memory is being used, what model is running, how the runtime is configured, and where every reported value comes from, rather than only reporting a single total. It then analyzes the workload and projects what different optimization strategies, starting with quantization, would do to memory use before any change is made.
A process view breaks memory down into weights, KV cache, workspace and, with an embedded attach mode inside the inference process, activations. Every section is labeled either measured from a live source or unavailable with a reason, and only the optimization analysis is labeled projected; it never mutates the running model.
It runs on any NVIDIA GPU machine, laptop, workstation, cloud VM or bare-metal server, and installs with pip; embedded deep metrics need an additional install in the same Python environment as the model.