# Frontier Vision-Language Models: Architectural Evolution, Benchmarks, Applications, and Challenges

A survey and continuously updated collection of vision-language model papers and models, tracking the architectural evolution of VLMs across eras from contrastive alignment to unified multimodal transformers.

This repository is a survey and continuously updated collection of vision-language model (VLM) papers and models. It tracks the architectural evolution of VLMs through four eras: early models with frozen vision and language towers bridged by a connector, models that use a pretrained large language model as the trunk with vision as an adapter, models that early-fuse all modalities into a single transformer, and emerging world-action models that predict and act on their environment.

Within the newer eras it distinguishes models that only fuse inputs and generate text from omni-modal models that also generate images, video or speech through additional decoder heads, citing examples of each architecture as they appear.

It is maintained as a running reference for keeping up with new VLM releases and their benchmarks, applications and open challenges.

Website: https://github.com/zli12321/Vision-Language-Models-Overview