Fine-Tuning Vision Language Model (Qwen2.5-VL-7B) with TRL
A step-by-step tutorial notebook for fine-tuning the Qwen2.5-VL-7B vision-language model with Hugging Face TRL for visual question answering on the ChartQA dataset.
Visit website (opens in a new tab)github.com
This tutorial walks through fine-tuning Qwen2.5-VL-7B, a vision-language model, using the Hugging Face TRL library, adapting the pretrained model for visual question answering on chart images.
It uses the ChartQA dataset, which pairs chart images with questions and answers for visual reasoning tasks, and is designed to run in Google Colab with GPU support, or locally with an A100-class GPU and at least 16 GB of GPU memory.
The repository is a Jupyter notebook plus setup instructions, targeting Python 3.8 or newer with a CUDA-compatible PyTorch installation.