GPT-QModel
An extensible LLM quantization toolkit supporting GPTQ, AWQ, GGUF, FP8, and other formats across NVIDIA, AMD, Huawei Ascend, and Intel hardware, integrated with Transformers, vLLM, and SGLang.
GPT-QModel is an extensible platform for LLM quantization, validation, and deployment, supporting formats including GPTQ, AWQ, ParoQuant, GGUF, FP8, EXL3, and QQQ across NVIDIA CUDA, AMD ROCm, Huawei Ascend, Intel XPU, and CPU, integrated with Transformers, vLLM, and SGLang. Recent releases have added quantization support for a growing list of model families and reliability improvements such as device-aware kernel selection, safer CUDA dispatch, correct calibration masks, and hardened runtime caching.
The library loads Prism and Bonsai GGUF checkpoints through its own native GGUF loading path and internal runtime shim, so no external gguf package from PyPI is required, and models load through a small Python API that mirrors familiar Hugging Face conventions.
It is written in Python and released without an explicit stated license (NOASSERTION).