Evaluation & BenchmarksSurge AI

Surge AI publishes LMArena is a cancer on AI

Published "LMArena is a cancer on AI" — reporting disagreement with 52% of 500 votes in its own sample, and strong disagreement with 39%

Why it matters

A critique that leaderboards are optimized for formatting and impressions rather than accuracy. But this is the company's own analysis as a seller of human data, not an independent check

Source — Primary sources first; secondary reports are labelled “Press”

Officialsurgehq.aihttps://surgehq.ai/blog/lmarena-is-a-plague-on-ai

Added in Week 35, 2026 · See that week

Same week

Ming Wu and Pengyuan Zhu release Agent Zero MemoryPaperAutomated researchers can reliably mitigate alignment failuresOfficialSliding-window beats linear attentionPaperResearchers release RealSWEPaper

Nearby on the same axis

Google DeepMind and partners pilot double-blind evaluation of frontier modelsOfficialResearchers release RealSWEPapervLLM publishes optimization results for SemiAnalysis's AgentX benchmarkOfficialVerifiable Social ReasoningPaper