SIE
A self-hosted, open-source inference engine that serves every model an AI agent needs, such as search, document parsing, structured output and safety, from one OpenAI-compatible cluster.
SIE (Superlinked Inference Engine) is an open-source inference engine that runs the models behind every agent task through one API, including search and retrieval, document-to-markdown conversion, structured output, content safety, and the translation, transcription and vision calls around an agent loop. It replaces a separate model server per task with one system that serves over 100 models, loading each on demand.
It exposes an OpenAI-compatible API, including embeddings, chat completions, completions and responses endpoints, for drop-in migration, with a pre-configured catalog of models such as Stella, SPLADE, Qwen3, GLiNER and SigLIP, benchmarked on MTEB for embedding and retrieval quality. It serves multiple models at once with on-demand loading and least-recently-used eviction, ships Kubernetes and Helm deployment configs with KEDA autoscaling and Grafana dashboards, and integrates with LangChain, LlamaIndex, Haystack, DSPy, CrewAI, Chroma, Qdrant, Weaviate and LanceDB.
It is written in Python and released under the Apache 2.0 license.