Social Simulation Arena
A live, public benchmark where entrants forecast what a population will do before a real release happens, with forecasts hashed and timestamped before the outcome exists and scored once it is published.
Social Simulation Arena is a live benchmark for social simulation. Before each scheduled release, registered entrants forecast how a population will respond, such as how it will answer a poll or what it will search for. Forecasts lock 48 hours before the real answer publishes; at that point every forecast's hash is submitted to OpenTimestamps and proven in a Bitcoin block, so no one, including the arena's operators, can see the answer first.
Entering requires an entrant id, one HTTPS endpoint that receives a signed JSON question and returns a forecast in the requested type, such as a number, a profile or a ranking, and a pull request to register. Number questions are scored by CRPS, profile questions by the energy score, and rankings by rank-biased overlap, with an overall score set so persistence scores 0 and a perfect oracle scores 100.
The repository includes the site itself, an example agent endpoint and a probe tool for testing an endpoint locally before registering.