LatentGate
A VL-JEPA-inspired pipeline that compresses images, text, conversations and RAG documents locally through Ollama before sending only the compact result to a cloud LLM API, cutting token costs by around 80 percent.
LatentGate runs vision and text compression locally through Ollama, at no API cost, and then forwards only the compacted payload to a cloud model such as GPT-4o, Claude or Gemini, instead of sending a full image or long prompt directly. Its token optimizer reports around 66 percent fewer tokens on a realistic development corpus in about a millisecond, and its selective-decoding mode uses cosine similarity between video frames to call the cloud API only when a scene actually changes.
It works as an MCP server for Claude Desktop, Cursor, Cline and Zed, exposes a Python API and a FastAPI REST server, and supports OpenAI, Anthropic, Google, Groq, DeepSeek, Together, Azure, AWS Bedrock and Ollama or any OpenAI-compatible endpoint as the downstream provider. Every saving is measured with a real tokenizer and checked for lost facts, and cost tracking, structured logging and Docker deployment are included.