LLM Applications
A step-by-step guide and interactive notebook for building and scaling a retrieval-augmented generation LLM application for production, from ingesting documents through serving and evaluation.
LLM Applications is a comprehensive guide to building RAG-based LLM applications for production, paired with an interactive notebook. It walks through developing a retrieval-augmented generation application from scratch, scaling its major components such as loading, chunking, embedding, indexing and serving, and evaluating different configurations against both per-component and overall quality scores.
It also covers implementing a hybrid routing approach between open-source and closed LLMs to bridge cost and quality gaps, and serving the finished application in a scalable, highly available way, plus the first- and second-order impacts the authors observed from deploying LLM applications in their own products.
It uses OpenAI's chat models and Anyscale Endpoints for open-source models such as Llama 2, and is built to run on Ray, with setup instructions for both a local machine and an Anyscale workspace with GPU worker nodes.