Local LLM with RAG
An experimental Streamlit application in which a locally run Ollama model decides for itself when to search a folder of the user's documents, rather than always retrieving text on a fixed schedule.
This project is a sandbox for testing agentic retrieval-augmented generation with local models. Built with Ollama and Pydantic AI, the agent itself decides when and how to search the user's documents instead of following a fixed retrieval pipeline, using Ollama to create embeddings with nomic-embed-text and storing them in LanceDB for vector search.
A Streamlit interface lets the user pick a model, point at a document folder, and chat with the documents; the required models download automatically on first run. Because the agent calls a search tool itself, the selected model must support tool calling, and the project documents which Ollama models were benchmarked for this on a small evaluation set graded by a larger model acting as judge, including which model families handle tool calling unreliably for this task.