Jan Server
A self-hosted, microservices-based LLM API platform with an OpenAI-compatible chat endpoint, multi-step tool orchestration, MCP tool integration, and OAuth authentication, deployable through Docker Compose or Kubernetes.
Jan Server provides an OpenAI-compatible API for chat completions, a separate Response API for multi-step tool orchestration, a Media API with S3 integration, and Model Context Protocol tools for web search, scraping and code execution. Authentication runs through Keycloak over OAuth/OIDC with guest access, and the platform includes observability through OpenTelemetry, Prometheus, Jaeger and Grafana.
It is set up through an interactive command-line wizard that walks through choosing a local vLLM or remote OpenAI-compatible model provider, an MCP search provider such as Serper or SearXNG, and whether to enable the Media API, then starts every service through Docker Compose; a scripted setup is also available. Running it needs Docker, at least 8 GB of RAM, and an NVIDIA GPU only if local vLLM inference is used.
It is written in Go.