hf-mem
A lightweight Python CLI that estimates the inference memory a Hugging Face model needs, reading only its Safetensors or GGUF metadata over HTTP range requests rather than downloading the model.
hf-mem estimates the inference memory requirements for any model on the Hugging Face Hub that ships Safetensors or GGUF weights, including Transformers, Diffusers and Sentence Transformers models. It stays lightweight by depending only on httpx2 and pulling metadata through HTTP range requests instead of downloading the model itself, and is recommended to run with uv.
It can be run directly as a CLI with uvx hf-mem --model-id, or called programmatically from Python with a synchronous run() function or an asynchronous arun() for use inside an async application, both returning a result with the estimated memory, KV cache size and total memory.
An experimental flag adds KV cache memory estimation for causal language models and vision-language models, with configurable maximum model length, batch size and KV cache data type, plus a weights-only breakdown of the base model and combined experts for mixture-of-experts models. The project is still experimental and may have breaking changes before a 1.0 release.