Keystone
Keystone is a clinical decision-evidence benchmark, built on HealthBench, that tests whether a chat assistant's recommended action changes correctly when the one fact it depends on is removed, contradicted or altered.
Keystone extends HealthBench-style clinical chat benchmarking, which normally scores a reply against a physician-written rubric for a fixed message, by testing what an assistant does when the decisive fact behind that message changes. Every source conversation is annotated with a decision frame, the decision being made, the action the evidence supports, and the facts it rests on, then paired with edited twins that change exactly one thing and a paraphrase-only control that changes nothing.
The release covers 7,318 twins built from 1,236 HealthBench conversations across seven physician-agreed strata, with four edit families applied to every source (removing a decisive fact, adding a contradicting fact, changing an attribute that should change the advice, and adding an irrelevant fact as a negative control) and four more families applied where the source allows. A judge from a different vendor than the one being evaluated maps each reply to its annotation, so outcomes are scored by action rather than wording, and the physician rubric's own applicability is scored alongside the model.