Hugging Face explains the MMLU score mismatch on the Open LLM Leaderboard
MMLU score mismatch on the Open LLM Leaderboard explained — the same LLaMA 65B scored 0.636/0.637/0.488 depending on the implementation
Why it matters
The event that publicly confirmed benchmark numbers are not comparable
Source — Primary sources first; secondary reports are labelled “Press”
Officialhuggingface.cohttps://huggingface.co/blog/evaluating-mmlu-leaderboardAdded in Week 35, 2026 · See that week