Expert analysis and in-depth reviews of machine learning research papers. Covering computer vision, deep learning, and AI innovations with practical insights.
Woosuk Kwon, Zhuohan Li, +7
How PagedAttention (the memory manager behind vLLM) applies OS-style virtual-memory paging to the KV cache — fixed-size blocks, a block table, and copy-on-write prefix sharing — to eliminate fragmentation and dramatically raise LLM serving throughput.
William Fedus, Barret Zoph, +1
How the Switch Transformer scales to trillions of parameters by routing each token to a single expert (top-1 gating), decoupling model capacity from per-token compute while managing expert load with a capacity factor.
Alex Krizhevsky, Ilya Sutskever, +1
How AlexNet won ImageNet 2012 by a landslide and launched the deep-learning era: a deep convolutional network trained on two GPUs with ReLU activations, dropout regularization, and aggressive data augmentation.
Chao Jia, Yinfei Yang, +8
How ALIGN scales vision-language pre-training to 1.8 billion noisy image-alt-text pairs, showing that the sheer scale of a raw, uncurated dataset can outweigh careful data cleaning for a simple dual-encoder contrastive model.
Jacob Devlin, Ming-Wei Chang, +2
How BERT pre-trains a deep bidirectional Transformer with masked language modeling and next-sentence prediction, then fine-tunes the same model to state-of-the-art results across a wide range of NLP tasks.
Junnan Li, Dongxu Li, +2
How BLIP unifies vision-language understanding and generation in one model, and bootstraps noisy web data with CapFilt — a captioner that synthesizes captions and a filter that removes noisy image-text pairs.
Jiahui Yu, Zirui Wang, +4
How CoCa trains a single image-text foundation model with both a contrastive loss and a generative captioning loss by splitting the text decoder into unimodal and multimodal halves, computed in one forward pass.
Jean-Baptiste Alayrac, Jeff Donahue, +25
How Flamingo bridges a frozen vision encoder and a frozen language model with a Perceiver Resampler and gated cross-attention layers, enabling few-shot learning from interleaved image-text sequences.
Bin Xiao, Haiping Wu, +7
How Florence-2 turns detection, segmentation, grounding, captioning, and OCR into one sequence-to-sequence problem driven by task prompts, expressing spatial outputs as location tokens in the text vocabulary.