Back to directory

ToolGuard

A security and reliability layer for AI agent tool calls that intercepts MCP and other tool traffic through a seven-layer interceptor, flags prompt injection and schema drift, and runs automated fuzz tests against an agent's tools in CI.

ToolGuard

ToolGuard sits between an AI agent and the tools it calls, checking each call against a seven-layer semantic and schema interceptor before it reaches the tool. A real-time dashboard shows blocked events with the JSON payload and a timeline of the call chain, useful for demos and for reviewing why a call was blocked after the fact; a separate terminal mode runs the same checks headlessly for CI pipelines, Docker containers or quick local iteration.

The project is explicitly scoped to what the authors call execution reliability - tool calls, argument types, chaining and API responses - rather than judging whether an agent's reasoning is good; that is left to separate evaluation frameworks. A drift-check command compares a tool's current JSON schema against an earlier snapshot and can fail a CI build if the schema silently changed, and a replay command re-runs a previously captured failure for debugging.

It integrates with LangChain and CrewAI-based agents and is tested with pytest.

More in AI evals

See all 269
  • Provenance gate

    github.com

    A deterministic authorization gateway that sits between an AI agent and its tools, tracing where each parameter of a side-effect call came from, and blocking the call or asking a human when the source does not check out.

  • JevBench

    benchmarkheaven.com

    A reproducible benchmark from Benchmark Heaven that ranks Jev-class typed decision models on intelligence, calibration, speed, and cost, with a published harness, scoring rules, and results.

  • A co-evolutionary adversarial red-teaming tool for AI agents that uses quality-diversity evolutionary search to autonomously breed edge-case scenarios, testing whether an agent with tool access can be made to hallucinate, deadlock or leak funds.