Back to directory

NAGI BENCH

A benchmark site that runs the same prompt through different combinations of model, agent harness and thinking effort, then shows the resulting runnable artifacts side by side for comparison.

NAGI BENCH

NAGI BENCH is a collection of one-shot evaluation cases for AI coding agents. Each case gives one prompt to different combinations of a model, an agent harness such as Claude Code, Codex CLI or Cursor, and a thinking-effort setting, then displays the resulting runnable artifacts next to each other for comparison.

The project treats a model paired with a harness as a single unit under test, since the harness controls tool calls, system prompts and context handling as much as the model does. Each combination is evaluated at its default, out-of-the-box behavior, without extra skills, plugins or custom system prompts, so results stay comparable across agents.

A registry table generated from the project's own data, kept in sync by continuous integration, tracks which models and harnesses have been tested and how many outputs each combination has produced.

More in AI evals

See all 269
  • Provenance gate

    github.com

    A deterministic authorization gateway that sits between an AI agent and its tools, tracing where each parameter of a side-effect call came from, and blocking the call or asking a human when the source does not check out.

  • JevBench

    benchmarkheaven.com

    A reproducible benchmark from Benchmark Heaven that ranks Jev-class typed decision models on intelligence, calibration, speed, and cost, with a published harness, scoring rules, and results.

  • A co-evolutionary adversarial red-teaming tool for AI agents that uses quality-diversity evolutionary search to autonomously breed edge-case scenarios, testing whether an agent with tool access can be made to hallucinate, deadlock or leak funds.