Prehistory — From Neural Networks to the TransformerJonas Gehring et al. (Facebook AI Research)FAIR

Convolutional Sequence to Sequence Learning (ConvS2S) released on arXiv

Convolutional Sequence to Sequence Learning (ConvS2S) released on arXiv

Why it matters

Performed translation with convolutions alone, showing the race to move beyond recurrence just before the Transformer

Source — Primary sources first; secondary reports are labelled “Press”

Paperarxiv.orghttps://arxiv.org/abs/1705.03122

Added in Week 35, 2026 · See that week

Nearby on the same axis

In-Datacenter Performance Analysis of a Tensor Processing UnitPaperZhu et al. publish CycleGAN on arXivPaperFinn, Abbeel and Levine publish Model-Agnostic Meta-Learning (MAML) on arXivPaperArjovsky, Chintala and Bottou publish Wasserstein GAN on arXivPaper