MechVQA

MechVQA is a benchmark of 3,300 high-density mechanical engineering drawings and 21,000 question-answer pairs across ten fine-grained tasks in three capability levels, recognition, reasoning and judging, built through a semi-automated construction and quality-control pipeline to test how well multimodal large language models read orthographic multi-view projections, dimensioning, section views and symbolic notation. General-purpose multimodal models are shown to be brittle on this material, due to dense annotations, weak domain priors and unreliable spatial-relation reasoning under strict projection rules.

The accompanying MechVL model is a domain-specialized baseline trained through a multi-stage pipeline of supervised fine-tuning followed by two-stage self-play reinforcement learning, reaching a total score of 84.85 on the benchmark and outperforming the strongest closed-source multimodal models tested. Inference code, the reinforcement-learning training framework, the evaluation pipeline and benchmark, a data-generation pipeline, and a compact fine-tuning recipe are available now; model checkpoints and training data are released through ModelScope and Hugging Face.