# ATM-Bench

A benchmark for evaluating whether an AI agent or memory system can answer personal, evidence-grounded questions about roughly four years of a person's own multimodal history, including images, videos and emails.

ATM-Bench measures long-term, personalized, referential memory: questions that refer to something the user has previously said, seen or done, which the answering system must retrieve and ground in the right piece of evidence rather than guess. It ships baseline implementations for retrieval methods and several ported third-party memory systems, and separately benchmarks general-purpose coding and agent systems, including Claude Code, Codex, OpenCode, Pi, OpenClaw and Antigravity, run through per-question, isolated evaluation harnesses, with an accompanying leaderboard and per-run cost estimates.

A harder subset of the benchmark is distributed as a Harbor dataset, so any Harbor-compatible agent can be run against a fixed set of difficult questions in per-question isolated containers.

It is written in Python and released under the MIT license, accompanying a published research paper.

Website: https://github.com/JingbiaoMei/ATM-Bench