Prehistory — From Neural Networks to the TransformerSepp Hochreiter
Hochreiter's TU Munich thesis identifies the vanishing gradient problem
Identified the vanishing gradient problem in a thesis at the Technical University of Munich
Why it matters
The first to show that the learning signal vanishes as depth and sequence length grow. The problem definition behind LSTM and residual connections
Source — Primary sources first; secondary reports are labelled “Press”
Pressen.wikipedia.orghttps://en.wikipedia.org/wiki/Vanishing_gradient_problemAdded in Week 35, 2026 · See that week