SDNQ

A cross-platform quantization engine for PyTorch models that reduces memory use and increases inference and training speed on GPUs, NPUs, CPUs and ARM devices.

Visit website github.com

SDNQ is a PyTorch-native quantization engine that runs on any device through an Eager-mode fallback, while using faster Inductor or OpenVINO-compiled paths on CUDA, ROCm, XPU, CPU and ARM hardware when available. It supports 176 storage types from 1 to 16 bits, including int, uint, floating point and unsigned floating point formats, and techniques such as Hadamard rotation, SVD quantization and Lloyd-Max codebook quantization on both weights and quantized matrix multiplication.

Beyond inference, SDNQ supports full-parameter quantized training, including quantized optimizers and direct updates to quantized weights, and it is natively supported by the upstream Diffusers library from version 0.40.0.

It is written in Python, installed with pip, and licensed under GPL-3.0.

More in Models

See all 405

mini-AGI
github.com
A continual learning language model that trains from scratch on a single 8 GB VRAM GPU, assembling its own architecture and adapting its size to the hardware and data available rather than shipping as a fixed pretrained model.
- Models
Applied Inference Lab: Compliance Control Embedding Stack
github.com
An open-source contrastive-embedding pipeline that maps technical CVE vulnerability descriptions to formal cybersecurity compliance controls in NIST SP 800-53 and CMMC, published with its model generation, fine-tuning and evaluation code.
- Models
- Security
peekaboolean
github.com
A small vision-language model, fine-tuned with LoRA on a 500-million-parameter backbone, that answers typed multiple-choice, ordinal-score and yes/no questions about an image with calibrated probabilities, in about 400 milliseconds on a laptop.
- Models
- Inference

More in Models

mini-AGI

Applied Inference Lab: Compliance Control Embedding Stack

peekaboolean

Cookies