Back to directory

Hermes Katana

A defense-in-depth security toolkit for LLM agents that tracks where every piece of text came from, scans it for prompt injection, applies policy before a tool runs, and scrubs outbound secrets.

Hermes Katana

Hermes Katana attaches a source, such as user input or fetched web content, to every character of text using a taint-tracking model inspired by Google DeepMind's CaMeL paper, so that when strings from different sources are combined, sliced or transformed, each resulting character still carries its origin and a tool call built from tainted, dangerous content can be denied automatically. Runtime policies, written in YAML, decide what happens for clean, tainted, dangerous or unknown-origin tool calls, including routing some calls to a human for approval.

Injection detection uses a small distilled MiniLM classifier by default, with a larger DeBERTa-v3-large model available for higher accuracy, and a Proving Ground harness runs empirical, multi-model attack-effectiveness tests with explicit gates against false positives and against regressions in known attacks. The base install is intentionally small and works without downloading any model, with larger model and dataset artifacts kept on Hugging Face rather than in the repository.

More in AI evals

See all 269
  • Provenance gate

    github.com

    A deterministic authorization gateway that sits between an AI agent and its tools, tracing where each parameter of a side-effect call came from, and blocking the call or asking a human when the source does not check out.

  • JevBench

    benchmarkheaven.com

    A reproducible benchmark from Benchmark Heaven that ranks Jev-class typed decision models on intelligence, calibration, speed, and cost, with a published harness, scoring rules, and results.

  • A co-evolutionary adversarial red-teaming tool for AI agents that uses quality-diversity evolutionary search to autonomously breed edge-case scenarios, testing whether an agent with tool access can be made to hallucinate, deadlock or leak funds.