MNIST-PRO: MNIST is Back as a Partially Observable World for AI Agents</p>\n","updatedAt":"2026-09-01T03:55:09.556Z","author":{"_id":"626b626405fe1cb65725aca1","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/626b626405fe1cb65725aca1/E-uD9h3n0lN04MPDbgkoH.png","fullname":"Soujanya Poria","name":"soujanyaporia","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":13,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8980182409286499},"editors":["soujanyaporia"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/626b626405fe1cb65725aca1/E-uD9h3n0lN04MPDbgkoH.png"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.31022","authors":[{"_id":"6a963d08cd6ebc484732ec57","name":"Vernon Toh","hidden":false},{"_id":"6a963d08cd6ebc484732ec58","name":"Navonil Majumder","hidden":false},{"_id":"6a963d08cd6ebc484732ec59","name":"Zhengyuan Liu","hidden":false},{"_id":"6a963d08cd6ebc484732ec5a","name":"Nancy F. Chen","hidden":false},{"_id":"6a963d08cd6ebc484732ec5b","name":"Soujanya Poria","hidden":false}],"publishedAt":"2026-08-31T00:00:00.000Z","submittedOnDailyAt":"2026-09-01T00:00:00.000Z","title":"MNIST-PRO: MNIST is Back as a Partially Observable World for AI Agents","submittedOnDailyBy":{"_id":"626b626405fe1cb65725aca1","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/626b626405fe1cb65725aca1/E-uD9h3n0lN04MPDbgkoH.png","isPro":false,"fullname":"Soujanya Poria","user":"soujanyaporia","type":"user","name":"soujanyaporia"},"summary":"AI agents in partially observable environments need to coordinate active sensing with working memory to maintain an evolving perceptual state. However, existing benchmarks struggle to isolate this perceptual-state construction and interpretation capability because they introduce physical and control complexities. We address this with MNIST-PRO, a benchmark that isolates agentic perception by converting MNIST digit recognition into a sequential, glimpse-based search task with lookback constraints. We evaluate ten multimodal models across four memory representations, including raw visual history, textual states, structured metric grid maps, and a consolidated visual canvas. While models excel under full observability, partial observability exposes a clear performance gap. We identify three distinct bottlenecks. First, perceptual-state construction and interpretation present a challenge, as agents struggle to integrate fragmented glimpses. Second, agents often stop exploring before they see the full sequence. Third, models often fail to revise early, incorrect beliefs even when faced with subsequent contradictory evidence. These results show that simply acquiring visual evidence is not enough. Agents must also be able to build and update a reliable perceptual state.","upvotes":3,"discussionId":"6a963d08cd6ebc484732ec5c","ai_summary":"The MNIST-PRO benchmark isolates perceptual-state construction in partially observable settings, revealing that multimodal agents struggle to integrate fragmented glimpses, continue exploring, and revise incorrect beliefs.","ai_keywords":["MNIST-PRO","active sensing","working memory","perceptual state","glimpse-based search","partial observability","multimodal models","memory representations","perceptual-state construction"],"ai_summary_model":"thinkingmachines/Inkling-Small","organization":{"_id":"626ab9dac804c432c1b27a48","name":"declare-lab","fullname":"Deep Cognition and Language Research (DeCLaRe) Lab","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/626b626405fe1cb65725aca1/pRDzr_NKUm9VhqSBqjHzW.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"633f875450d83f5065d459d6","avatarUrl":"/avatars/48312867ac2c05d589869972f90881b9.svg","isPro":false,"fullname":"Pala Tej Deep","user":"Tej3","type":"user"},{"_id":"638488b3be729f36b2ec46d4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/638488b3be729f36b2ec46d4/p0FJyzBKWlql6QJ7OtN_i.jpeg","isPro":false,"fullname":"Vernon Y.H. Toh","user":"vernontoh","type":"user"},{"_id":"63ac5701c21e60a3e9b58aa7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63ac5701c21e60a3e9b58aa7/g6EX7diOpuA94R2ab-rZC.png","isPro":true,"fullname":"Dipankar Sarkar","user":"dipankarsarkar","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"626ab9dac804c432c1b27a48","name":"declare-lab","fullname":"Deep Cognition and Language Research (DeCLaRe) Lab","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/626b626405fe1cb65725aca1/pRDzr_NKUm9VhqSBqjHzW.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.31022.md","query":{}}">

Get this paper in your agent:

Don't have the latest CLI?

Cite arxiv.org/abs/2608.31022 in a model README.md to link it from this page.

Cite arxiv.org/abs/2608.31022 in a dataset README.md to link it from this page.

Cite arxiv.org/abs/2608.31022 in a Space README.md to link it from this page.

Add this paper to a collection to link it from this page.