GGUF Parser
A command-line tool that reviews GGUF model files and estimates their memory usage and maximum tokens per second, reading only the file's metadata so nothing has to be downloaded first.
GGUF Parser reviews and estimates the resource usage of GGUF-format models, the binary format GGML-based executors such as llama.cpp use for fast model loading. It reads a model's metadata through chunked remote reads, so a file does not need to be downloaded or loaded to be checked.
It estimates memory usage, typically within about 100 MiB of the actual figure, and can calculate maximum tokens per second from supplied device metrics without running the model. It identifies what a GGUF file is for, such as an embedding, reranking or LoRA adapter model, and can estimate usage across multiple GPU devices, with configurable layer offload, context size, flash attention and memory-mapping settings.
It can parse a local file, a remote file, or a model referenced from Hugging Face, ModelScope or the Ollama library, and also estimates audio-projector models and models using sliding-window attention. It is written in Go.