Local LLM Stack Documentation
This project documents, step by step, how to assemble a self-hosted local LLM stack from open-source components: a chat interface, document embedding, a vector database, web search and several MCP servers.
Written by someone who describes themself as not an AI expert, the documentation walks through installing Docker and the NVIDIA container toolkit, then configuring each piece of the stack in turn: Portainer for managing containers, Ollama for serving models, Open WebUI as the chat front end, SearXNG for web search, Docling for document processing, vLLM as an additional model server, MCP servers and an MCPO adapter, Kokoro for text-to-speech, and Watchtower for keeping container images up to date.
The stack as a whole provides a web chat interface, document embedding and retrieval, a vector database, a relational database for configuration and history, user authentication, and GPU support, and ends with one combined Docker Compose file that brings all of it up together.
It is documented alongside a Python codebase and carries an explicit disclaimer that it reflects personal research rather than expert guidance.