ZxBench
A reproducible, self-hosted platform for benchmarking local large language models with versioned question sets, containerized programming tests, rule-based and optional AI-judge scoring, and auditable reports.
ZxBench is a locally deployed benchmarking platform for large language models. It runs a versioned question bank across eleven dimensions, including programming, hallucination resistance, reasoning and math, structured output, tool and CLI workflows, and safety and authority, with programming tasks executed in isolated, resource-limited Docker containers.
A run saves a snapshot of its question set, and progressive multi-step questions carry a model's own earlier answers forward within the same run. Scoring separates a run finishing from its score being trustworthy: environment failures, missing Docker images, and rule or AI-judge conflicts are recorded separately rather than silently counted as model failures, and a run whose scoring is not complete is kept out of the leaderboard. Batches of up to eight models can be evaluated together, with live monitoring supporting pause, resume, cancel and per-question retry, and a read-only command verifies a frozen run's scores after the fact.
It requires Node.js 22.13 or newer, pnpm, and a running Docker daemon, and is written in TypeScript under the MIT license.