Centralize company knowledge with an Enterprise RAG Chatbot
A QuickStart deployment that lets a team experiment with retrieval-augmented generation in a chat interface, uploading documents, tuning sampling parameters and switching between simple and agent-based RAG.
This QuickStart demonstrates retrieval-augmented generation for enterprise knowledge, using a fictional company scenario in which employees get answers to HR, procurement, sales and IT questions through a single chat interface. It lets a user explore the example solution, upload new documents to be embedded, adjust sampling parameters that influence the model's responses, use custom system prompts, and switch between a simple RAG pipeline and an agent-based one that can call additional tools.
Its documented architecture runs on OpenShift AI for container orchestration and GPU acceleration, using the LLaMA Stack framework to standardize the application's building blocks, a Streamlit chat interface, a Llama-3.2-3B-Instruct model for response generation, a safety guardrail model to block harmful requests and responses, Model Context Protocol servers for added functionality, an all-MiniLM-L6-v2 embedding model, and a PostgreSQL database with the PGVector extension for storing and searching embeddings, with Kubeflow Pipelines handling data ingestion.
It documents minimum hardware and software requirements and the user permissions the deployment needs, and is licensed under MIT.