VLM-Driving-Scene-Understanding
A QLoRA fine-tune of Qwen2.5-VL-3B that reads a driving-scene photo and outputs structured analysis of lane lines, vehicles, traffic signs and driving risk.
VLM-Driving-Scene-Understanding fine-tunes Qwen2.5-VL-3B-Instruct with QLoRA (4-bit NF4 quantization plus LoRA rank 8) so that, given a driving-scene image, it outputs four structured fields: lane lines, vehicles, traffic signs, and driving risk. Training uses the nuScenes CAM_FRONT dataset and a Gradio web demo serves the resulting model for interactive use, alongside a Python inference API for single or batch image analysis.
A later version replaced free-text model output with a schema-validated structured pipeline: each image now produces a record with lane, vehicle-count, traffic-sign and risk fields that are checked against a schema, with one automatic repair retry before a failure is recorded explicitly rather than stored as raw text.
It is written in Python with PyTorch and released under the MIT license.