Fit-to-Size Intelligent Tensor Quantization for GGUF
A deterministic planning layer that turns discrete llama.cpp GGUF quantization presets into a near-continuous model-size control, spending a fixed byte budget on the most useful tensor-level precision upgrades.
FIT-GGUF starts from the largest standard llama.cpp GGUF quantization preset that still fits under a requested byte budget, then spends the remaining bytes on deterministic, tensor-level precision upgrades, effectively turning a preset picker into a size slider. Because tensor and quantization-type transitions are discrete, a small amount of target slack can remain, but the tool's size predictor is required to exactly match the resulting artifact.
The project validates deterministic size prediction and recipe execution, reporting zero-byte actual-versus-predicted error across 22 evaluated targets on two model families under a pinned toolchain, while explicitly not claiming that its guided tensor allocator is universally optimal for quality, since it helped on one model family but did not beat matched random allocation on the second. Its first public release ships 14 size tiers from 7 to 13.5 GiB, in 0.5 GiB steps, for one specific base model, each with its plan, tensor override file, measured five-domain evaluation results, and SHA-256 provenance.
A newer version adds fidelity tiers that let a user choose a target quality level instead of a raw size.