Back to directory

Sediment

A self-hosted evidence store for coding agents that captures model calls, code changes, developer decisions and check results, so teams can compare models, reuse prior session context, and build training datasets.

Sediment

Sediment records what a coding agent did on a team's own infrastructure: model calls, the code changes that resulted, developer decisions and the results of automated checks. That evidence can be used to evaluate agent work by comparing models on recorded decisions, code retention and check outcomes, to reuse selected messages, tool calls and results from a previous session, or to export examples for fine-tuning, preference training or reinforcement learning.

It integrates with Claude Code, Codex, Cursor, the Pi agent and Copilot Chat, though the signals available vary by agent. Installation is a shell script on macOS with Homebrew, Debian or Ubuntu, followed by a local server that a team can also deploy to EC2 or their own host if they already run PostgreSQL and HTTPS.

It is licensed under AGPL-3.0, with the harness shims under the MIT license.

More in AI evals

See all 269
  • Provenance gate

    github.com

    A deterministic authorization gateway that sits between an AI agent and its tools, tracing where each parameter of a side-effect call came from, and blocking the call or asking a human when the source does not check out.

  • JevBench

    benchmarkheaven.com

    A reproducible benchmark from Benchmark Heaven that ranks Jev-class typed decision models on intelligence, calibration, speed, and cost, with a published harness, scoring rules, and results.

  • A co-evolutionary adversarial red-teaming tool for AI agents that uses quality-diversity evolutionary search to autonomously breed edge-case scenarios, testing whether an agent with tool access can be made to hallucinate, deadlock or leak funds.