OptiLLM
An OpenAI API-compatible inference proxy that applies more than 20 published inference-time techniques, such as mixture-of-agents and Monte Carlo tree search, to improve LLM accuracy on reasoning tasks without retraining.
OptiLLM is an inference proxy, compatible with any OpenAI-style API endpoint, that implements more than 20 published techniques for improving LLM accuracy and reasoning performance at inference time rather than through additional training or fine-tuning. Techniques range from a simple best-of-N sampling approach to more advanced methods such as Monte Carlo tree search and mixture-of-agents, selected by prefixing the requested model name.
It is a drop-in replacement for an existing OpenAI-compatible client: a request only needs its base URL pointed at the running proxy, with no other code changes. It supports OpenAI, Anthropic, Google, Cerebras and more than 100 other models through LiteLLM, and reports accuracy gains on benchmarks such as AIME, Math-L5, GPQA-Diamond and Arena-Hard-Auto.