Harbor
A framework from the creators of Terminal-Bench for evaluating and optimizing coding agents and language models across thousands of parallel sandboxed environments.
Harbor is a framework from the creators of Terminal-Bench for evaluating and optimizing agents and language models. It can evaluate arbitrary agents such as Claude Code, OpenHands, or Codex CLI, build and share custom benchmarks and environments, run experiments across thousands of environments in parallel through providers such as Daytona, Modal, LangSmith, Blaxel, Novita Sandbox, Tensorlake, and Runta, and generate rollouts for reinforcement learning optimization.
It installs as a Python package and serves as the official harness for running Terminal-Bench-2.0, launching benchmarks locally through Docker by default or on a cloud provider by passing an environment flag. It can also list and run other third-party benchmarks, such as SWE-Bench and Aider Polyglot, through the same command-line interface.