llm-bench-rig
A dual-engine benchmarking pipeline for GGUF and safetensors language models on NVIDIA GPUs, measuring both speed and answer quality with a live dashboard and exportable benchmark cards.
llm-bench-rig is a benchmarking pipeline for GGUF and safetensors language models on NVIDIA GPUs, built for the RTX 5090 but working with any CUDA-capable card. It runs speed benchmarks through llama.cpp or the vLLM API across several prompt and context lengths, including a served HTTP mode that measures time-to-first-token and streaming throughput as percentiles.
Quality benchmarks run five tasks, including MMLU, ARC-Challenge and HumanEval, through the project's own generative evaluators rather than the lm-evaluation-harness, so scores can differ from harness numbers by a few points but stay internally consistent; a confidence interval is computed per task so close results can be read as a tie. Every run records a provenance block with the server build, chat template hash and configuration, and results export as HTML reports and image cards. It needs Python 3.10 and a CUDA-capable GPU.