Pretraining Large Language Models with NVFP4 published
Pretraining Large Language Models with NVFP4 published — a 12B model trained on 10T tokens (the longest 4-bit training run in public documents), matching the FP8 baseline in training loss and accuracy
Why it matters
The strongest positive evidence that FP4 training actually works. However, it is the vendor's own report, and no independent reproduction has been confirmed
Source — Primary sources first; secondary reports are labelled “Press”
Paperarxiv.orghttps://arxiv.org/abs/2509.25149Added in Week 35, 2026 · See that week