26.9%\n🔥 Partial: 54.8 -> 61.6%\n💸 ~9× cheaper: ~$7.8 vs ~$72/task, 224K -> 100K tokens\n\nThe same harness lifts Opus 4.8 across 5 benchmarks:\n🚀 OSWorld-Verified 80.9 -> 81.9\n🚀 WindowsAgentArena 41.6 -> 50.6\n🚀 AndroidWorld 69.0 -> 81.9\n🚀 MobileWorld 51.3 -> 70.1\n\nAnd it's not tied to Claude. Drop in our internal 31B SFR-CUA and it leaps across the board:\n🚀 OSWorld 2.0 7.6 -> 43.2\n🚀 OSWorld-Verified 66.9 -> 81.1\n🚀 WindowsAgentArena 40.9 -> 51.2\n🚀 AndroidWorld 68.1 -> 84.1\n🚀 MobileWorld 48.7 -> 68.4\nA small model, punching far above its weight.\n\n💡 Computer-use was never only about vision. It's an agent reasoning problem, tying together what it sees, the state it acts on, and the plan it holds across hundreds of steps.","html":"<p>🚨 We just set a new SOTA on OSWorld 2.0!<br>⭐️ And we did it by making our computer-use agent look less, not more.</p>\n<p>Everyone's racing the same way: better vision, sharper screen-reading, more pixels. We went the opposite direction. StateAct barely looks at the screen at all.</p>\n<p>The insight nobody wanted to say out loud: a screenshot is a lie. It's a lossy picture of what actually matters, the files, backends, and DOM behind the glass. Pixels can't tell them apart. Code can.</p>\n<p>So we made program state the primary interface, and call a GUI specialist only when a subgoal is truly visual, just 1.1% of steps.</p>\n<p>The result, on the exact same Claude Opus 4.8:<br>🔥 Binary: 20.6 -&gt; 26.9%<br>🔥 Partial: 54.8 -&gt; 61.6%<br>💸 ~9× cheaper: ~$7.8 vs ~$72/task, 224K -&gt; 100K tokens</p>\n<p>The same harness lifts Opus 4.8 across 5 benchmarks:<br>🚀 OSWorld-Verified 80.9 -&gt; 81.9<br>🚀 WindowsAgentArena 41.6 -&gt; 50.6<br>🚀 AndroidWorld 69.0 -&gt; 81.9<br>🚀 MobileWorld 51.3 -&gt; 70.1</p>\n<p>And it's not tied to Claude. Drop in our internal 31B SFR-CUA and it leaps across the board:<br>🚀 OSWorld 2.0 7.6 -&gt; 43.2<br>🚀 OSWorld-Verified 66.9 -&gt; 81.1<br>🚀 WindowsAgentArena 40.9 -&gt; 51.2<br>🚀 AndroidWorld 68.1 -&gt; 84.1<br>🚀 MobileWorld 48.7 -&gt; 68.4<br>A small model, punching far above its weight.</p>\n<p>💡 Computer-use was never only about vision. It's an agent reasoning problem, tying together what it sees, the state it acts on, and the plan it holds across hundreds of steps.</p>\n","updatedAt":"2026-07-28T01:44:08.410Z","author":{"_id":"6090ff099a8bcaa437b234a4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6090ff099a8bcaa437b234a4/iUvw7JXT-ngI7rGk1x-io.jpeg","fullname":"Ziyang Luo","name":"Ziyang","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":27,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8615548014640808},"editors":["Ziyang"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/6090ff099a8bcaa437b234a4/iUvw7JXT-ngI7rGk1x-io.jpeg"],"reactions":[{"reaction":"🔥","users":["HideOnBush","Ziyang","finniu"],"count":3}],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2607.22798","authors":[{"_id":"6a67fb6773f69d5af2bec51c","name":"Yan Yang","hidden":false},{"_id":"6a67fb6773f69d5af2bec51d","user":{"_id":"636865b8cca0a0a962c21f3f","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Mja7cpws4gb2Jmdj_foPA.png","isPro":true,"fullname":"Xiangru (Edward) Jian","user":"HideOnBush","type":"user","name":"HideOnBush"},"name":"Xiangru Jian","status":"claimed_verified","statusLastChangedAt":"2026-07-28T08:45:04.528Z","hidden":false},{"_id":"6a67fb6773f69d5af2bec51e","user":{"_id":"6090ff099a8bcaa437b234a4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6090ff099a8bcaa437b234a4/iUvw7JXT-ngI7rGk1x-io.jpeg","isPro":false,"fullname":"Ziyang Luo","user":"Ziyang","type":"user","name":"Ziyang"},"name":"Ziyang Luo","status":"claimed_verified","statusLastChangedAt":"2026-07-28T08:45:04.523Z","hidden":false},{"_id":"6a67fb6773f69d5af2bec51f","name":"Zirui Zhao","hidden":false},{"_id":"6a67fb6773f69d5af2bec520","name":"Yutong Dai","hidden":false},{"_id":"6a67fb6773f69d5af2bec521","name":"Ziji Shi","hidden":false},{"_id":"6a67fb6773f69d5af2bec522","name":"Hanshu Yan","hidden":false},{"_id":"6a67fb6773f69d5af2bec523","name":"Jun Hao Liew","hidden":false},{"_id":"6a67fb6773f69d5af2bec524","name":"Silvio Savarese","hidden":false},{"_id":"6a67fb6773f69d5af2bec525","name":"Junnan Li","hidden":false}],"publishedAt":"2026-07-24T00:00:00.000Z","submittedOnDailyAt":"2026-07-28T00:00:00.000Z","title":"StateAct: Program State, before Pixels, for Long-Horizon Computer-Use Agents","submittedOnDailyBy":{"_id":"6090ff099a8bcaa437b234a4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6090ff099a8bcaa437b234a4/iUvw7JXT-ngI7rGk1x-io.jpeg","isPro":false,"fullname":"Ziyang Luo","user":"Ziyang","type":"user","name":"Ziyang"},"summary":"Computer-use agents are usually improved by strengthening perception: better models for reading a screenshot and choosing where to click. Yet a screenshot is only a lossy rendering of the underlying program state, e.g., the files, application backends, and DOM that hold the task data. Different states can produce the same pixels, while code can inspect and modify that state directly. StateAct is a code-first, multi-agent harness built around this distinction. Its main agent works directly with program state by using code, while a dedicated GUI subagent handles screenshot-and-click interaction on the few subgoals that need it, just 28 of 108 tasks and 1.1% of main-agent steps. The same direct access to program state also supports verification: an independent finish gate double-checks the saved result for structural failures, e.g., output that is missing, unsaved, or written to the wrong path. To stay on track over hundreds of steps, the main agent hands subgoals to fresh subagents, keeping its own context focused. On OSWorld 2.0, StateAct lifts Claude Opus 4.8 from 20.6% to 26.9% on binary success, and from 54.8% to 61.6% on partial success, at ~ 9x lower cost per task than the same model driven by screenshots alone; a code-only variant with no GUI subagent reaches only 45.9% partial, below that screenshot-based baseline's 54.8%. In general, grounding action, verification, and memory in state, what we call state-grounding, shifts the main bottleneck from perception toward reasoning: failures depend more on what the agent thinks than on what it sees.","upvotes":51,"discussionId":"6a67fb6773f69d5af2bec526","organization":{"_id":"5f6d64475e78cc6b0ed31e4c","name":"Salesforce","fullname":"Salesforce AI Research","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1602756670970-noauth.jpeg"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6090ff099a8bcaa437b234a4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6090ff099a8bcaa437b234a4/iUvw7JXT-ngI7rGk1x-io.jpeg","isPro":false,"fullname":"Ziyang Luo","user":"Ziyang","type":"user"},{"_id":"636865b8cca0a0a962c21f3f","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Mja7cpws4gb2Jmdj_foPA.png","isPro":true,"fullname":"Xiangru (Edward) Jian","user":"HideOnBush","type":"user"},{"_id":"65164444bc0631719873af81","avatarUrl":"/avatars/0e68ea5b5369273a07e5889480ca9421.svg","isPro":false,"fullname":"Wei Pang","user":"weipang142857","type":"user"},{"_id":"6452d79149b6b9a2383b5775","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/T28lP0kE7PZIGzJjhSpSx.jpeg","isPro":false,"fullname":"Tianyu Zhang","user":"TianyuZhang","type":"user"},{"_id":"66a06c4815b87c4a072fa4da","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/HMKq6icy7pSfYK7lwoeck.png","isPro":false,"fullname":"Hanshu YAN","user":"hanshuyan","type":"user"},{"_id":"6615538f231480986e6faf3c","avatarUrl":"/avatars/f6b12f380aac0abdf36e1ab57ecda2f5.svg","isPro":false,"fullname":"Tianyu Zhang","user":"TianyuZhang-ServiceNow","type":"user"},{"_id":"644d15c26dfd5f8240d9a437","avatarUrl":"/avatars/47ba51e1b2968aa6477ac08fe583d692.svg","isPro":false,"fullname":"Yunshun Zhong","user":"NeilZhong","type":"user"},{"_id":"66c9b8a03864174f35a51b28","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/0tUoi-ddcDH6mgNLrfgEk.png","isPro":false,"fullname":"hao xu","user":"moreerom","type":"user"},{"_id":"69cbcaf7b2d7df49df84fe57","avatarUrl":"/avatars/ce7d1cfbe72329b936c91ab8cf10ed17.svg","isPro":false,"fullname":"ZHU YINUO","user":"roger333555","type":"user"},{"_id":"65befcba2e67686440065e99","avatarUrl":"/avatars/7b087e4b5e3319b92df00646955dcc88.svg","isPro":false,"fullname":"Chao Zhang","user":"CC2ChiChi","type":"user"},{"_id":"69bbdd39157bf58b1398b86f","avatarUrl":"/avatars/14a86e723e2cbf4bac0e03e810981f10.svg","isPro":false,"fullname":"Lanxin Zhao","user":"Kriscamp","type":"user"},{"_id":"6891bbff78946201296b4592","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6891bbff78946201296b4592/ECmWBrlfeonPg0HzmQ_sW.png","isPro":false,"fullname":"Yuqing Li","user":"MindscapeRAG","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"5f6d64475e78cc6b0ed31e4c","name":"Salesforce","fullname":"Salesforce AI Research","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1602756670970-noauth.jpeg"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2607/2607.22798.md","query":{}}">

Get this paper in your agent:

Don't have the latest CLI?

Cite arxiv.org/abs/2607.22798 in a model README.md to link it from this page.

Cite arxiv.org/abs/2607.22798 in a dataset README.md to link it from this page.

Cite arxiv.org/abs/2607.22798 in a Space README.md to link it from this page.

Add this paper to a collection to link it from this page.