Model Architecture & Training MethodsWortsman et al.

Model Soups

Model Soups

Why it matters

Demonstrated that simply averaging the weights of several fine-tuned models raises performance; the starting point of model merging

Source — Primary sources first; secondary reports are labelled “Press”

Paperarxiv.orghttps://arxiv.org/abs/2203.05482

Added in Week 35, 2026 · See that week

Same week

Marcus writes Deep Learning Is Hitting a Wall for NautilusOfficialmuP / Tensor Programs VPaper

Nearby on the same axis

muP / Tensor Programs VPaperChinchillaPaperGoogle Research announces PaLM (540B)OfficialLi et al. present Diffusion-LMPaper