# Thinking with Visual Primitives — PyTorch Implementation

An unofficial PyTorch reproduction of DeepSeek's method for teaching multimodal LLMs to reason with bounding boxes and points interleaved directly in their chain of thought.

This project implements a multi-stage training pipeline that teaches multimodal LLMs to reason with bounding boxes and points as first-class "thought units," interleaving spatial coordinates within chain-of-thought text to close what the authors call the Reference Gap in visual reasoning.

Training runs in three stages: pretraining a base model to output the visual-primitive format, specialized supervised fine-tuning of separate box-grounding and point-grounding experts, and on-policy distillation of both experts into one unified model. Because of compute constraints, every stage in this reproduction uses LoRA fine-tuning rather than full-parameter training, so the results are presented as a feasibility check rather than a state-of-the-art claim.

Pretrained checkpoints for each stage, built on Qwen2VL-2B, are published on Hugging Face. It is written in Python with PyTorch and is MIT licensed.

Website: https://github.com/vra/Thinking-with-Visual-Primitives-pytorch