Back to directory

SIMURG

A streaming integrity monitor that detects decoding corruption in an LLM's output, such as repetition loops or language drift, while the answer is still generating, so the host can cut the stream and regenerate before the user sees it.

SIMURG

SIMURG (Streaming Integrity Monitor & Universal Regeneration Guard) watches a language model's token stream character by character and looks for a statistical signature of decoding corruption: repetition collapse, cross-lingual drift, regurgitated boilerplate, structural breakdown into symbol garbage, or leaked chat-template tokens. This is distinct from factual hallucination, which the project says leaves no such statistical trace.

It measures signals such as repetition rate, lexical variety, script distribution, compressibility and predictive surprise in real time, and can abort a corrupted stream within a few hundred characters of onset so the host can regenerate the answer before a user sees it. It runs far faster than typical LLM token generation, needs only numpy with no model or GPU, and is set up in a few lines of code with no training step; its false-alarm rate is configurable and calibrated with conformal prediction. It works with any OpenAI-compatible API.

The project also includes a free web search tool for agents, a live dashboard for watching the guard in action, and optional deep-learning and self-healing tiers.

More in AI evals

See all 269
  • Provenance gate

    github.com

    A deterministic authorization gateway that sits between an AI agent and its tools, tracing where each parameter of a side-effect call came from, and blocking the call or asking a human when the source does not check out.

  • JevBench

    benchmarkheaven.com

    A reproducible benchmark from Benchmark Heaven that ranks Jev-class typed decision models on intelligence, calibration, speed, and cost, with a published harness, scoring rules, and results.

  • A co-evolutionary adversarial red-teaming tool for AI agents that uses quality-diversity evolutionary search to autonomously breed edge-case scenarios, testing whether an agent with tool access can be made to hallucinate, deadlock or leak funds.