Deploying LLMs locally
Training material that shows how to deploy and run large language models on local infrastructure such as HPC systems, including a slide deck, source code, and model and data download scripts.
This repository is training material illustrating how to use AI tools and techniques on local infrastructure, including HPC systems. It contains a PowerPoint presentation on running large language models on a local machine, a source-code directory with accompanying examples, a models directory with scripts to download pre-trained models, a data directory with scripts to download datasets, and a tools directory for running LLMs locally.
Because machine-learning dependencies for different topics can conflict, the project uses a separate Conda environment per subdirectory in source-code rather than one shared environment, to keep dependency management simple and avoid version conflicts between scripts.