GPT-5.5 Scores 10.6% on ActiveVision, Humans Hit 96.1% [R]
Mirrored from r/MachineLearning for archival readability. Support the source by reading on the original site.
The interesting finding from a new [arXiv paper](https://arxiv.org/abs/2607.16165) isn't that a frontier vision model failed a new benchmark, that happens weekly, but the specific shape of the failure and the fact that the models cannot patch it by writing their own code.
The benchmark, called ActiveVision, contains 17 tasks across 3 categories designed, in the authors' words, to "force repeated visual perception rather than a single static description." GPT-5.5 at the highest exposed reasoning-effort tier solves 10.6% of items and scores zero on 11 of the 17 tasks. Claude Fable 5, which the authors note tops most reasoning and coding leaderboards, manages 3.5%. Three human participants averaged 96.1%.
[link] [comments]
More from r/MachineLearning
-
A collision-entropy floor for watermark/retrieval AI-text detection. Looking for a sanity check before I take this further [D]
Aug 14
-
Are supervised and unsupervised learning still relevant today? [D]
Aug 14
-
TMLR Relevance and Prestige [D]
Aug 13
-
Reproducible canvas-aligned low-level patterns in somerandomllm-generated images and their possible relation to iterative editing artifacts [D]
Aug 13
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.