pxa
A quantization codec and kernel pack, plus an auto-tuning launcher, for running large language models efficiently on older NVIDIA Pascal and Volta cards that lack DP4A support.
pxa provides the PXQ codec and matching kernels for cards such as the Tesla P100 and V100, and the project publishes a growing set of PXQ-quantized models, including dense and mixture-of-expert variants, that others can download and run directly. A launcher script inspects the NVIDIA cards and model files present on a machine, asks what the session is for, and then chooses the engine, batch sizes, tensor split, flash-attention setting, chat template and environment automatically, showing the reasoning behind each choice before starting.
Benchmark results comparing pxa against llama.cpp on the same cards and settings are published alongside each release, and a set of twelve step-by-step tutorials, shipped inside the release archive as well as online, cover a first chat reply through quantizing a custom model, running a companion vLLM instance, and troubleshooting.