Awesome MLLM for Code
A curated list of research papers on using multimodal large language models to generate code from non-text inputs, covering web and mobile UI screenshots, scientific plots, slides, SVG logos, CAD, posters and games.
This repository collects papers on code generation under multimodal scenarios, where a model must produce code from an image or diagram rather than only from a text description. Categories include web and mobile UI code generation from screenshots or designs, scientific plot code generation, visually rich programming problems that include image examples, SVG code generation, slide code generation, program repair under these same multimodal scenarios, UML and CAD code generation, poster and multimodal document generation, 3D code generation, and game code generation.
A further section covers general benchmarks and foundation multimodal code models, and one covers using code itself as a way to test a multimodal model's image understanding. Each entry links to the paper's title, authors, venue and year, and the list accompanies a 2026 TMLR paper surveying the same area.