AI Infrastructure SoftwareJay Shah et al. (Colfax, Together AI, Meta, NVIDIA, Princeton)MetaNVIDIAPrinceton University
FlashAttention-3 published
FlashAttention-3 published — Hopper-specific asynchrony delivers 1.5–2.0× in FP16 on H100 (up to 740 TFLOPs/s), and about 1.2 PFLOPs/s in FP8
Why it matters
A case showing that kernels must be rewritten for a specific generation of silicon. Quantitatively rebutted accuracy concerns about FP8 attention
Source — Primary sources first; secondary reports are labelled “Press”
Paperarxiv.orghttps://arxiv.org/abs/2407.08608Added in Week 35, 2026 · See that week