Explore machine learning papers and reviews related to Contrastive Learning. Find insights, analysis, and implementation details.
Chao Jia, Yinfei Yang, +8
How ALIGN scales vision-language pre-training to 1.8 billion noisy image-alt-text pairs, showing that the sheer scale of a raw, uncurated dataset can outweigh careful data cleaning for a simple dual-encoder contrastive model.
Junnan Li, Dongxu Li, +2
How BLIP unifies vision-language understanding and generation in one model, and bootstraps noisy web data with CapFilt — a captioner that synthesizes captions and a filter that removes noisy image-text pairs.
Jiahui Yu, Zirui Wang, +4
How CoCa trains a single image-text foundation model with both a contrastive loss and a generative captioning loss by splitting the text decoder into unimodal and multimodal halves, computed in one forward pass.
Xiaohua Zhai, Basil Mustafa, +2
How SigLIP replaces CLIP’s softmax contrastive loss with a pairwise sigmoid loss, removing the need for a global all-gather normalization and enabling strong language-image pre-training at small batch sizes.
Jean-Bastien Grill, Florian Strub, +12
How self-supervised learning works without negative pairs — a predictor and momentum target network are all you need to prevent representation collapse.
Kaiming He, Haoqi Fan, +3
How a momentum-updated encoder and a dictionary queue make contrastive learning practical — large dictionaries with consistent keys, no large-batch requirement.
Ting Chen, Simon Kornblith, +2
How a simple framework — augmentation, shared encoder, projection head, and contrastive loss — set a new standard for self-supervised visual representation learning.
Adrien Bardes, Jean Ponce, +1
How variance, invariance, and covariance regularization enables self-supervised representation learning without negative pairs or momentum encoders.