Back to directory

NVIDIA NeMo Relay

A multi-language runtime that adds visibility and policy control over an agent's tool and LLM calls without modifying the existing agent stack, exporting events and traces to observability tools.

NVIDIA NeMo Relay

NVIDIA NeMo Relay gives coding agents, applications, framework integrations, and observability backends a shared runtime for tracking scopes, enforcing policy, running plugins, and reacting to lifecycle events, without requiring changes to the agent code it observes. A quick-start path records a real Codex or Claude Code run on disk, writing both raw events and a normalized trajectory file that can be inspected and built on.

It is designed to complement OpenTelemetry's GenAI conventions and existing observability or evaluation products rather than replace them, and can export its own event and trajectory formats alongside standard OpenTelemetry traces, logs, and metrics. It has built-in integrations with LangChain, LangGraph, and Deep Agents, and is built directly into Hermes Agent, which needs no separate Relay setup. It installs as a prebuilt CLI from PyPI or a platform installer script, and is Apache 2.0 licensed.

More in AI evals

See all 269
  • Provenance gate

    github.com

    A deterministic authorization gateway that sits between an AI agent and its tools, tracing where each parameter of a side-effect call came from, and blocking the call or asking a human when the source does not check out.

  • JevBench

    benchmarkheaven.com

    A reproducible benchmark from Benchmark Heaven that ranks Jev-class typed decision models on intelligence, calibration, speed, and cost, with a published harness, scoring rules, and results.

  • A co-evolutionary adversarial red-teaming tool for AI agents that uses quality-diversity evolutionary search to autonomously breed edge-case scenarios, testing whether an agent with tool access can be made to hallucinate, deadlock or leak funds.