Evaluation & BenchmarksFutureHouse
Humanity's Last Exam bio/chem error rate put at 29%; HLE's own review finds 18%
Raised a 29% error rate in the biology and chemistry questions of Humanity's Last Exam; the HLE team's own review put it at 18%, and it announced rolling revisions
Why it matters
A case that showed, through public refutation and response, that the answer keys of frontier benchmarks can themselves be wrong
Source — Primary sources first; secondary reports are labelled “Press”
Officialfuturehouse.orghttps://www.futurehouse.org/research/hle-examAdded in Week 35, 2026 · See that week