# ComfyUI Quantization Toolkit

A ComfyUI extension that quantizes diffusion models to low-bit formats such as W4A4, W4A8 and W8A8, loads pre-quantized checkpoints and quantization-aware LoRAs, and tunes inference for low-VRAM GPUs.

ComfyUI Quantization Toolkit quantizes diffusion models used in ComfyUI workflows down to 4-bit or 8-bit weight and activation formats, either converting an already-loaded floating-point or FP8 model on the fly or loading a checkpoint that was quantized ahead of time. It supports standard, stochastic-requantized and dynamic-runtime modes for applying LoRAs to a quantized model, and can export a model in its native quantized format.

It adds nodes for enabling quantization on a loaded model, loading a quantized model directly, and applying a quantized LoRA stack, along with dynamic VRAM controls and lazy Torch Compile support for reducing memory use during inference. It requires ComfyUI 0.32.0 or newer, a compatible comfy-kitchen package, an NVIDIA GPU with useful INT8 throughput, and installs either through the ComfyUI Manager or by cloning into the custom_nodes directory. It was formerly published as ComfyUI-INT8-Toolkit.

Website: https://github.com/SparknightLLC/ComfyUI-QuantizationToolkit