omnicache-ai
A multi-layer caching library for AI agent pipelines that sits in front of LLM calls, embeddings and retrieval queries across frameworks such as LangChain, LangGraph, AutoGen, CrewAI and Agno, caching by semantic similarity as well as exact match.
omnicache-ai is a caching library for AI agent pipelines, designed to sit in front of any LLM call, embedding request, retrieval query or agent workflow step to avoid repeating identical or semantically similar work. It caches LLM responses keyed by model and message content, stores computed embedding vectors for reuse, and caches retrieval results by query and top-k so that semantically similar questions can return a cached answer through cosine-similarity matching rather than only exact matches.
It ships thirteen framework adapters, including LangChain, LangGraph, AutoGen, CrewAI and Agno, an adaptive semantic similarity threshold, a streaming cache for streamed responses, a tiered backend combining an in-process layer with a persistent store such as Qdrant or Weaviate, stampede protection, multi-tenant namespacing, and Prometheus and OpenTelemetry metrics export. The project describes itself as an actively maintained alternative to GPTCache, which it says has been effectively unmaintained since 2024.
It is written in Python and MIT licensed.