Recent advancements in Vision-Language Models (VLMs) showcase key trends and innovations:

Video and Interleaved Formats: 

VLMs now process interleaved text-video-image formats, with some models handling videos independently.
Examples: Idefics3, Llava-Next 
Interleave (interleaved), LongVU, Video-Llava (video-specific).

Multiple Vision Encoders: 

Models combine outputs from multiple vision encoders, selectively fusing representations for better multimodal understanding.
Examples: BRAVE, MiniGemini, DocOwl.

Retrieval with VLMs: 

Dual-encoder setups pair VLMs with LLMs, projecting outputs to maximize similarity for retrieval tasks.
Examples: ColPali, ColQwen, DSE, MCDSE.

Zero-Shot Vision Tasks: 

VLMs achieve zero-shot performance in tasks like object detection, segmentation, and counting.
Examples: KOSMOS-2 (detection), PaliGemma (segmentation via token-to-mask mappers), Molmo (counting)

Training Strategies: 

New pre- and post-training methods include domain-specific setups. GOT-OCR exemplifies document-focused training, aligning vision encoders with domain shifts for improved accuracy.