Scale AI releases SWE-Bench Pro
SWE-Bench Pro released — 41 repositories and 1,865 tasks, in three tiers: public, held-out and commercial
Why it matters
A follow-up design that responds to contamination and saturation in the SWE-bench family. An attempt to prevent leakage structurally by separating tiers
Source — Primary sources first; secondary reports are labelled “Press”
Paperarxiv.orghttps://arxiv.org/abs/2509.16941Added in Week 35, 2026 · See that week