Microsoft presents ZeRO-Infinity
Presented ZeRO-Infinity — uses heterogeneous GPU+CPU+NVMe memory, exceeding 25 petaflops on 512 V100 GPUs
Why it matters
Got around the GPU memory wall by extending into the storage tier. Established the framing that the constraint is bandwidth, not memory
Source — Primary sources first; secondary reports are labelled “Press”
Paperarxiv.orghttps://arxiv.org/abs/2104.07857Added in Week 35, 2026 · See that week