PyTorch 2.9 released

PyTorch 2.9 released — a wider stable libtorch ABI for third-party C++/CUDA extensions, symmetric memory for writing multi-GPU kernels, an error/resume toggle for torch.compile graph breaks, wheel variants for ROCm, XPU and CUDA 13, and FlexAttention enabled on Intel GPUs. 452 contributors and 3,216 commits since 2.8

Why it matters

A direction that lets users write multi-GPU kernels directly at the framework API level. On top of NCCL-style collective communication, user kernels gain one more path

Source — Primary sources first; secondary reports are labelled “Press”

Officialpytorch.orghttps://pytorch.org/blog/pytorch-2-9/

Added in Week 36, 2026 · See that week

Same week

NVFP4 support requested in llama.cpp Discussion #16668OfficialKarpathy appears on Dwarkesh PodcastPressIn the same interview, Karpathy says nanochat was not built by vibe codingPressAnthropic launches Agent Skills (SKILL.md, progressive disclosure)Official

Nearby on the same axis

Helion released in betaOfficialPyTorch releases MonarchOfficialPretraining Large Language Models with NVFP4 publishedPaperNVIDIA releases NCCL 2.28Official