cascadeflow
A runtime library that makes per-step model, budget and policy decisions inside an AI agent's execution loop, aiming to cut cost and latency while preserving output quality.
cascadeflow is a runtime intelligence layer for AI agents that makes cost, latency, quality, budget and compliance decisions inside the agent's execution loop, rather than at the boundary of an HTTP proxy. It works per model call, tool call and sub-agent handoff, drawing on accumulated results from earlier calls so its routing improves as an agent runs.
It reports large cost savings on benchmarks such as MT-Bench, GSM8K, MMLU and TruthfulQA while retaining most of a flagship model's quality, and integrates with agent frameworks including LangChain, OpenAI Agents SDK, CrewAI, PydanticAI, Google ADK, n8n, Vercel AI SDK and Hermes Agent. Supported runtime actions include allowing a call, switching models, denying a tool call, or stopping execution, based on the current context and policy.
It installs as a Python package or an npm package, and is described as adding low overhead to each step.