# HashHop Long Context Evaluation

A benchmark generator that creates multi-hop reasoning problems over shuffled hash pairs, used to evaluate how well a language model's long-context architecture retrieves and chains facts across very large prompts.

HashHop builds synthetic evaluation data for testing long-context reasoning: its evaluation function generates a problem containing a large set of shuffled hash-to-hash pairs, then asks for one or more query hashes to be resolved through a chain of a configurable number of hops.

Parameters control the size of the problem in characters, the number of queries, the number of hops per query, the length of each hash, and whether the model should show its chain of reasoning explicitly. The output separates a prompt for evaluation or training, a completion suitable for training a model on the task, and a structured target mapping for scoring.

It is a small Python package installed with Poetry, released under the MIT license, and used as the benchmark behind the project's own long-context architecture research.

Website: https://magic.dev/blog/100m-token-context-windows