how much visual compute you throw at it.\n\n\n\n\n[X post](https://x.com/m_pulkit/status/2076839055640150474)","html":"<p>Your eyes don't see a whole scene at once — they dart around in a sequence of foveated glimpses. Modern vision models take in the whole image in one shot. That difference decides if a model can generalize to scenarios that are out-of-distribution.</p>\n<p><a href=\"https://cdn-uploads.huggingface.co/production/uploads/648a1075e8bee5332919e0ab/r4r6yqLVbkSec15s-XtIj.gif\" rel=\"nofollow\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/648a1075e8bee5332919e0ab/r4r6yqLVbkSec15s-XtIj.gif\" alt=\"Adobe Express - ACzVXh45Nj4LA_b8\"></a></p>\n<p>We built simple visual puzzles (e.g. \"read these switches, navigate, track the state\") and made them longer at test time than in training. SOTA VLMs — GPT-5.4, Claude Sonnet 4.6, Qwen — ace the short ones… then fall off a cliff the moment the puzzle gets longer. A tiny recurrent agent (green) extrapolates the performance at test time. Even with task-specific training, the global model (Qwen) still breaks out-of-distribution.</p>\n<p><a href=\"https://cdn-uploads.huggingface.co/production/uploads/648a1075e8bee5332919e0ab/haw5xrN8qgr9zSgpCH1hd.png\" rel=\"nofollow\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/648a1075e8bee5332919e0ab/haw5xrN8qgr9zSgpCH1hd.png\" alt=\"image\"></a></p>\n<p>The failure has a name: <strong>global shortcuts.</strong><br>When a model sees the whole image at once, it can memorize a parallel \"trick\" that works for the lengths it saw — but doesn't actually implement the step-by-step computation. So it breaks out-of-distribution. Same thing happens to LLMs on parity/state-tracking. We show vision has the same problem.</p>\n<p>You might think making the model recurrent (giving it a memory that updates step by step) is the fix. It isn't — on its own. Same recurrent LSTM, three ways of <em>seeing</em> the image:</p>\n<ul>\n<li><strong>Global</strong> — sees the whole image at once → collapses </li>\n<li><strong>Local + Global</strong> — adds small high-res crops but keeps the global view → still collapses</li>\n<li><strong>Foveated</strong> — only small local glimpses, no global view → generalized out-of-distribution.<br>Hand a recurrent net the whole image and it <em>still</em> memorizes a shortcut. <strong>Locality is a necessary ingredient.</strong></li>\n</ul>\n<p><a href=\"https://cdn-uploads.huggingface.co/production/uploads/648a1075e8bee5332919e0ab/H29jy3E70UGZAByM5FvNV.png\" rel=\"nofollow\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/648a1075e8bee5332919e0ab/H29jy3E70UGZAByM5FvNV.png\" alt=\"image\"></a></p>\n<p>Now flip it: fix the <em>local</em> glimpses, swap only the backbone. Strict recurrent nets (LSTM/GRU/RNN) generalize. Transformers, Mamba, xLSTM — all degrade. So the recipe is <strong>both</strong>: local perception + recurrence. Neither alone is enough. Together they're sufficient. 🔑</p>\n<p><a href=\"https://cdn-uploads.huggingface.co/production/uploads/648a1075e8bee5332919e0ab/ZEciJvAFuuC2NmMGbwatN.png\" rel=\"nofollow\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/648a1075e8bee5332919e0ab/ZEciJvAFuuC2NmMGbwatN.png\" alt=\"image\"></a></p>\n<p>There's a real trade-off. Small/low-res glimpses → generalize but slow to explore. Big/high-res glimpses → easy to explore but invite shortcuts. With the right glimpse settings, we show that FoveAgentLSTM holds its accuracy at resolutions well beyond training — a global model only works in/near the resolution it was trained at.</p>\n<p><a href=\"https://cdn-uploads.huggingface.co/production/uploads/648a1075e8bee5332919e0ab/wFn_RVTGTqfTGADq7BqDH.png\" rel=\"nofollow\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/648a1075e8bee5332919e0ab/wFn_RVTGTqfTGADq7BqDH.png\" alt=\"image\"></a></p>\n<p>But locality + recurrence isn't a free win everywhere — and knowing <em>when</em> it helps is the point. On a <strong>recall</strong> task the global VLM <em>wins</em> and our local agent lags. State-tracking wants recurrence + locality. Pure retrieval doesn't. The two split exactly like they do in language models.</p>\n<p><a href=\"https://cdn-uploads.huggingface.co/production/uploads/648a1075e8bee5332919e0ab/ez6fFfLB1BVcjGzRJjOzn.png\" rel=\"nofollow\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/648a1075e8bee5332919e0ab/ez6fFfLB1BVcjGzRJjOzn.png\" alt=\"image\"></a></p>\n<p>The synthetic tasks isolate the mechanism. Does it carry over to a real task? We tested reasoning over math plots — finding a function's roots. At the <strong>same visual-compute budget</strong>, a foveated Qwen adds <strong>+29 pts (~100%)</strong> of accuracy over the global baseline. Uniformly cranking resolution 10× buys almost nothing (+3.8 pts). Same lesson as the synthetic tasks: locality + recurrence beats brute-force scaling. How you spend visual compute &gt; how much visual compute you throw at it.</p>\n<p><a href=\"https://cdn-uploads.huggingface.co/production/uploads/648a1075e8bee5332919e0ab/w8jext-ThQgMiH8oB5sGC.png\" rel=\"nofollow\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/648a1075e8bee5332919e0ab/w8jext-ThQgMiH8oB5sGC.png\" alt=\"image\"></a></p>\n<p><a href=\"https://x.com/m_pulkit/status/2076839055640150474\" rel=\"nofollow\">X post</a></p>\n","updatedAt":"2026-07-18T01:33:28.162Z","author":{"_id":"648a1075e8bee5332919e0ab","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/648a1075e8bee5332919e0ab/KDzi904zGPuy_PL3gbUYE.jpeg","fullname":"M.Reza Ebrahimi","name":"rezaebrahimi","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":1,"identifiedLanguage":{"language":"en","probability":0.823442280292511},"editors":["rezaebrahimi"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/648a1075e8bee5332919e0ab/KDzi904zGPuy_PL3gbUYE.jpeg"],"reactions":[],"isReport":false}},{"id":"6a5af959a5f7ad08ccddbc4b","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":376,"isUserFollowing":false},"createdAt":"2026-07-18T03:56:09.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [Binding Visual Features Point by Point](https://huggingface.co/papers/2605.25427) (2026)\n* [VisReflect: Latent Visual Reflection for Fine-Grained Perception in Long Visual Context](https://huggingface.co/papers/2606.30288) (2026)\n* [VisualNeedle: Benchmarking Active Visual Search in Information-Dense Scenes](https://huggingface.co/papers/2605.26380) (2026)\n* [The Last Visible Pixel: Probing Fine-Scale Perception in Vision-Language Models](https://huggingface.co/papers/2606.07861) (2026)\n* [Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoning](https://huggingface.co/papers/2605.28160) (2026)\n* [Segmentation before Answering: Pixel Grounding for MLLM Visual Reasoning](https://huggingface.co/papers/2607.05798) (2026)\n* [Are Reasoning Vision-Language Models Robust to Semantic Visual Distractions?](https://huggingface.co/papers/2606.08894) (2026)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"<p>This is an automated message from the <a href=\"https://huggingface.co/librarian-bots\">Librarian Bot</a>. I found the following papers similar to this paper. </p>\n<p>The following papers were recommended by the Semantic Scholar API </p>\n<ul>\n<li><a href=\"https://huggingface.co/papers/2605.25427\">Binding Visual Features Point by Point</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2606.30288\">VisReflect: Latent Visual Reflection for Fine-Grained Perception in Long Visual Context</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.26380\">VisualNeedle: Benchmarking Active Visual Search in Information-Dense Scenes</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2606.07861\">The Last Visible Pixel: Probing Fine-Scale Perception in Vision-Language Models</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.28160\">Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoning</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2607.05798\">Segmentation before Answering: Pixel Grounding for MLLM Visual Reasoning</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2606.08894\">Are Reasoning Vision-Language Models Robust to Semantic Visual Distractions?</a> (2026)</li>\n</ul>\n<p> Please give a thumbs up to this comment if you found it helpful!</p>\n<p> If you want recommendations for any Paper on Hugging Face checkout <a href=\"https://huggingface.co/spaces/librarian-bots/recommend_similar_papers\">this</a> Space</p>\n<p> You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: <code>@librarian-bot recommend</code></p>\n","updatedAt":"2026-07-18T03:56:09.790Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":376,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.6890081167221069},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2607.09061","authors":[{"_id":"6a563143a9d74d6e65bbdcd4","name":"Pulkit Madan","hidden":false},{"_id":"6a563143a9d74d6e65bbdcd5","name":"Sanjay Haresh","hidden":false},{"_id":"6a563143a9d74d6e65bbdcd6","name":"Reza Ebrahimi","hidden":false},{"_id":"6a563143a9d74d6e65bbdcd7","name":"Sunny Panchal","hidden":false},{"_id":"6a563143a9d74d6e65bbdcd8","name":"Apratim Bhattacharyya","hidden":false},{"_id":"6a563143a9d74d6e65bbdcd9","name":"Roland Memisevic","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/648a1075e8bee5332919e0ab/lVV2rMkpgQRtEDok1ou38.gif"],"publishedAt":"2026-07-10T00:00:00.000Z","submittedOnDailyAt":"2026-07-17T00:00:00.000Z","title":"On Locality and Length Generalization in Visual Reasoning","submittedOnDailyBy":{"_id":"648a1075e8bee5332919e0ab","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/648a1075e8bee5332919e0ab/KDzi904zGPuy_PL3gbUYE.jpeg","isPro":false,"fullname":"M.Reza Ebrahimi","user":"rezaebrahimi","type":"user","name":"rezaebrahimi"},"summary":"A striking feature of the human visual system is that it ingests visual information through a series of local foveated glimpses, rather than a single global computation. This makes human vision distinctly different from most popular computer vision models in use today, which input images globally and in a single shot. A natural question therefore is whether local, sequential vision models may provide any fundamental computational benefits in addition to being biologically more plausible than global models. In this work, we investigate this question from the perspective of visual state tracking and length generalization. Inspired by recent studies of length generalization in language models, we study the behavior of vision models trained on simple vision tasks that require the aggregation of local information across an image. Our experiments reveal that, similar to language models, vision models can learn to exploit global shortcuts and thereby fail to generalize over task length or complexity. We also show that recurrent vision policies based on strictly local perception can mitigate these failures, thereby allowing models to generalize on these tasks. Our results show that local attention may be an essential overlooked requirement for robust compositional generalization.","upvotes":2,"discussionId":"6a563143a9d74d6e65bbdcda","organization":{"_id":"616851aa840fa49535b3d5b2","name":"qualcomm","fullname":"Qualcomm","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/65c2710dc79c1a6e4d22734d/kB7OTGVsC1DPMIV2femsf.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"698f899e0574510b757afb51","avatarUrl":"/avatars/1fc257e8924b25e3889f37ff3d5b42ad.svg","isPro":false,"fullname":"Hsg8l24mya3","user":"hsg8l24mya3","type":"user"},{"_id":"6a2da6c8ca070ee12c6e396c","avatarUrl":"/avatars/0355287dcabaa67dbc7f0b10b87451f9.svg","isPro":false,"fullname":"Joe Mama","user":"JoeMama123123123","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"616851aa840fa49535b3d5b2","name":"qualcomm","fullname":"Qualcomm","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/65c2710dc79c1a6e4d22734d/kB7OTGVsC1DPMIV2femsf.png"},"query":{}}">

Cite arxiv.org/abs/2607.09061 in a model README.md to link it from this page.

Cite arxiv.org/abs/2607.09061 in a dataset README.md to link it from this page.

Cite arxiv.org/abs/2607.09061 in a Space README.md to link it from this page.

Add this paper to a collection to link it from this page.