GGUF connector
A dependency-free Python package for reading, writing and dequantizing GGUF model files, with a bundled graphical and command-line interface for chatting with a loaded model.
gguf-connector provides the foundational code for working with GGUF, the model file format that succeeded GGML, without relying on external dependencies for its core read, write and dequantize operations. On top of that base, it also serves as a small graphical or command-line application that uses ctransformers or llama.cpp to load a GGUF chat model from the current directory and generate responses.
It installs through pip, an npm wrapper, or a shell or PowerShell install script, and its command-line interface adds optional modes for a vision-model connector, a metadata reader, a tensor reader that needs PyTorch, a PDF loader that needs pypdf, and a speech-recognition mode.