Back to directory

TokenOps

Run-aware token budget governance for multi-agent systems, enforcing a spending limit before every model call rather than reporting cost after the fact.

Visit website (opens in a new tab)theagentplane.github.io
TokenOps

TokenOps sets a single token budget for an entire agent workflow and enforces it before every step, so spending on many small, individually cheap calls does not quietly add up to a large bill. Ten configurable policies ship by default, and more can be added.

It requires Python 3.10 or newer and a running control plane that tracks spend for one or more agent processes; a demo command launches a throwaway plane for trying it out. Integrating it means wrapping an existing model call once and passing the wrapped version to an agent instead of the original; the wrapper checks the budget before each call and raises a Halt error when the run is out of budget, even from another process.

A Claude Code integration command wires the enforcement point in automatically; other agents follow a documented manual integration of about ten lines.

More in AI evals

See all 269
  • Provenance gate

    github.com

    A deterministic authorization gateway that sits between an AI agent and its tools, tracing where each parameter of a side-effect call came from, and blocking the call or asking a human when the source does not check out.

  • JevBench

    benchmarkheaven.com

    A reproducible benchmark from Benchmark Heaven that ranks Jev-class typed decision models on intelligence, calibration, speed, and cost, with a published harness, scoring rules, and results.

  • A co-evolutionary adversarial red-teaming tool for AI agents that uses quality-diversity evolutionary search to autonomously breed edge-case scenarios, testing whether an agent with tool access can be made to hallucinate, deadlock or leak funds.