Model Architecture & Training MethodsGoogle (Devlin et al.)Google

Google posts the BERT paper on arXiv

BERT paper posted on arXiv — pre-training a deep bidirectional Transformer encoder that conditions on both left and right context

Why it matters

Fine-tuned with just one additional output layer, it set new state-of-the-art results on 11 NLP tasks, including GLUE 80.4% and SQuAD v1.1 F1 93.2 (as reported by the authors)

Source — Primary sources first; secondary reports are labelled “Press”

Paperarxiv.orghttps://arxiv.org/abs/1810.04805

Added in Week 41, 2026 · See that week

Nearby on the same axis

OpenAI publishes the original GPT (GPT-1) paperOfficialFast Transformer DecodingPaperSparsely-Gated Mixture-of-Experts LayerPaperGShardPaper