Language Models

BERT: Pre-training of Deep Bidirectional Transformers

Jacob Devlin, Ming-Wei Chang, +2

How BERT pre-trains a deep bidirectional Transformer with masked language modeling and next-sentence prediction, then fine-tunes the same model to state-of-the-art results across a wide range of NLP tasks.