Back to directory

GGUF tools

A C library and command-line tool for inspecting and manipulating GGUF model files, showing metadata and tensors, comparing two models for similarity, and inspecting raw tensor weights.

GGUF tools

GGUF tools is a library and command-line program for working with GGUF files, the format used by llama.cpp and other local inference tools. Its show subcommand displays a file's key-value metadata, including arrays, and detailed information about every tensor, with offsets given relative to the start of the file rather than the data section.

Its compare subcommand checks whether two GGUF models are related, for instance whether one is a fine-tune of another, by computing the average weight difference for each matching tensor by name and parameter count, useful for seeing how much a model diverged and which layers were frozen during fine-tuning. Its inspect-tensor subcommand prints some or all of a named tensor's raw weight values, useful for checking quantization behavior or fingerprinting a model. An experimental subcommand extracts a smaller model from a mixture-of-experts file for experimentation, though the author notes the result does not perform useful work on its own.

It is written in C and licensed under MIT.

More in Inference

See all 770
  • OpenDecision

    deepanwadhwa.github.io

    An open-source Python package that answers typed questions about application state or documents using a local natural-language-inference model, returning structured choice, score, yes/no or evidence-relation values instead of free text.

  • Lifeboat

    iterate.ai

    A local inference server that puts an OpenAI- and Anthropic-compatible API in front of open-weight language models running on hardware you control, from a laptop to a GPU cluster.

  • Claude S40

    github.com

    An unofficial Claude chat client for 2007-era Nokia Series 40 phones, built as a Java ME app plus a small Go relay server, with web search, reading mode and saved chats.