Explore machine learning papers and reviews related to vision-languagemodels. Find insights, analysis, and implementation details.
How ALIGN scales vision-language pre-training to 1.8 billion noisy image-alt-text pairs, showing that the sheer scale of a raw, uncurated dataset can outweigh careful data cleaning for a simple dual-encoder contrastive model.
How BLIP unifies vision-language understanding and generation in one model, and bootstraps noisy web data with CapFilt — a captioner that synthesizes captions and a filter that removes noisy image-text pairs.
How CoCa trains a single image-text foundation model with both a contrastive loss and a generative captioning loss by splitting the text decoder into unimodal and multimodal halves, computed in one forward pass.
How Flamingo bridges a frozen vision encoder and a frozen language model with a Perceiver Resampler and gated cross-attention layers, enabling few-shot learning from interleaved image-text sequences.
How Florence-2 turns detection, segmentation, grounding, captioning, and OCR into one sequence-to-sequence problem driven by task prompts, expressing spatial outputs as location tokens in the text vocabulary.
How Qwen2-VL perceives images and video at any resolution with naive dynamic resolution (variable visual tokens) and M-RoPE, a multimodal rotary position embedding that decomposes position into temporal, height, and width components.
How SigLIP replaces CLIP’s softmax contrastive loss with a pairwise sigmoid loss, removing the need for a global all-gather normalization and enabling strong language-image pre-training at small batch sizes.