Skip to main content

Machine Learning Paper Reviews

89 reviews with explanations and code.

All Paper Reviews

Showing 46-54 of 89 papers

JMLR 2022

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

William Fedus, Barret Zoph, +1

How the Switch Transformer scales to trillions of parameters by routing each token to a single expert (top-1 gating), decoupling model capacity from per-token compute while managing expert load with a capacity factor.

ICML 2021

ALIGN: Scaling Vision-Language Learning with Noisy Text Supervision

Chao Jia, Yinfei Yang, +8

How ALIGN scales vision-language pre-training to 1.8 billion noisy image-alt-text pairs, showing that the sheer scale of a raw, uncurated dataset can outweigh careful data cleaning for a simple dual-encoder contrastive model.

TMLR 2022

CoCa: Contrastive Captioners are Image-Text Foundation Models

Jiahui Yu, Zirui Wang, +4

How CoCa trains a single image-text foundation model with both a contrastive loss and a generative captioning loss by splitting the text decoder into unimodal and multimodal halves, computed in one forward pass.

NeurIPS 2022

Flamingo: a Visual Language Model for Few-Shot Learning

Jean-Baptiste Alayrac, Jeff Donahue, +25

How Flamingo bridges a frozen vision encoder and a frozen language model with a Perceiver Resampler and gated cross-attention layers, enabling few-shot learning from interleaved image-text sequences.

Mastodon