MedLLM-Finetuning
An out-of-the-box fine-tuning framework, built on Hugging Face Transformers, PEFT, and DeepSpeed, for training large language models to perform binary classification on medical text such as disease-risk prediction.
MedLLM-Finetuning provides a ready-to-run pipeline for fine-tuning large language models on medical binary-classification tasks, such as predicting whether a patient will develop a condition from a text description. It supports QLoRA with 4-bit quantization, full-precision LoRA, and full fine-tuning, and integrates DeepSpeed with multi-GPU ZeRO optimization to reduce memory pressure during training.
Hyperparameters and dataset paths are defined in YAML configuration files separate from the training code, and the project documents a set of supported base models, including a small BioBERT encoder and larger medical and general-purpose models such as MedAlpaca, Meditron, Med42, Qwen2, Mistral, Llama 2 and 3, and Gemma 2. It requires a Linux machine with an NVIDIA GPU and a matching CUDA and PyTorch installation, and optionally supports FlashAttention-2 for faster training.