Evaluation & BenchmarksXuyang Liu et al.
OmniCompass released
OmniCompass released — a benchmark that measures video understanding by splitting it into three parts, when something appears, where it is and what is happening, with a separate track that uses sight only and a track that also uses sound
Why it matters
The authors said the vision track was built from 820 videos and 2,460 questions and the track that also uses sound from 240 videos and 720 questions, all annotated by people and checked several times, and that 50 models were evaluated. They reported that open models fell behind commercial models especially on tasks that use sound and screen together, and that answers often differed between pointing to the same object in text only and giving a picture along with it
Source — Primary sources first; secondary reports are labelled “Press”
Paperarxiv.orghttps://arxiv.org/abs/2610.18829Added in Week 43, 2026 · See that week