The rise of personal assistant agents, e.g., OpenClaw, highlights the growing potential of large language models to support users across everyday life and work. A core challenge in these settings is proactive assistance, since users often begin with underspecified requests and leave important needs, constraints, or preferences unstated. However, existing benchmarks rarely evaluate whether agents can identify and act on such hidden intents before they are explicitly stated, especially in sustained multi-turn interactions where user needs emerge gradually. To address this gap, we introduce π-Bench, a benchmark for proactive assistance comprising 100 multi-turn tasks across 5 domain-specific user personas. By incorporating hidden user intents, inter-task dependencies, and cross-session continuity, π-Bench evaluates agents’ ability to anticipate and address user needs over extended interactions, jointly measuring proactivity and task completion in long-horizon trajectories that better reflect real-world use. Experiments show (1) proactive assistance remains challenging, (2) a clear distinction between task completion and proactivity, and (3) the value of prior interaction for proactive intent resolution in later tasks.</p>\n","updatedAt":"2026-05-22T02:07:16.441Z","author":{"_id":"689ec537196ab997b13dc977","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/689ec537196ab997b13dc977/yXA_pd8ndjBIIg1Hx59QJ.png","fullname":"Haoran Zhang","name":"zzzhr97","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":3,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9074133038520813},"editors":["zzzhr97"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/689ec537196ab997b13dc977/yXA_pd8ndjBIIg1Hx59QJ.png"],"reactions":[{"reaction":"🔥","users":["rzzhan","xxxxxcritian","zzzhr97","greenboat11"],"count":4}],"isReport":false}},{"id":"6a0fc22093defbf011c68736","author":{"_id":"620f5a1c3f76c50e6458a9b6","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620f5a1c3f76c50e6458a9b6/pXh_f5F0UvufxuUa-eS-v.jpeg","fullname":"Peiyu Wang","name":"OrlandoHugBot","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":14,"isUserFollowing":false},"createdAt":"2026-05-22T02:40:32.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"nice work！\n","html":"<p>nice work！</p>\n","updatedAt":"2026-05-22T02:40:32.914Z","author":{"_id":"620f5a1c3f76c50e6458a9b6","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620f5a1c3f76c50e6458a9b6/pXh_f5F0UvufxuUa-eS-v.jpeg","fullname":"Peiyu Wang","name":"OrlandoHugBot","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":14,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.5807498693466187},"editors":["OrlandoHugBot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/620f5a1c3f76c50e6458a9b6/pXh_f5F0UvufxuUa-eS-v.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2605.14678","authors":[{"_id":"6a0e7c1a164dbbc68a26c56f","user":{"_id":"689ec537196ab997b13dc977","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/689ec537196ab997b13dc977/yXA_pd8ndjBIIg1Hx59QJ.png","isPro":false,"fullname":"Haoran Zhang","user":"zzzhr97","type":"user","name":"zzzhr97"},"name":"Haoran Zhang","status":"claimed_verified","statusLastChangedAt":"2026-05-21T19:22:15.982Z","hidden":false},{"_id":"6a0e7c1a164dbbc68a26c570","name":"Luxin Xu","hidden":false},{"_id":"6a0e7c1a164dbbc68a26c571","name":"Zhilin Wang","hidden":false},{"_id":"6a0e7c1a164dbbc68a26c572","name":"Runquan Gui","hidden":false},{"_id":"6a0e7c1a164dbbc68a26c573","name":"Shunkai Zhang","hidden":false},{"_id":"6a0e7c1a164dbbc68a26c574","name":"Haodi Lei","hidden":false},{"_id":"6a0e7c1a164dbbc68a26c575","name":"Zihao He","hidden":false},{"_id":"6a0e7c1a164dbbc68a26c576","name":"Bingsu He","hidden":false},{"_id":"6a0e7c1a164dbbc68a26c577","name":"Chicheng Qin","hidden":false},{"_id":"6a0e7c1a164dbbc68a26c578","name":"Tong Zhu","hidden":false},{"_id":"6a0e7c1a164dbbc68a26c579","name":"Xiaoye Qu","hidden":false},{"_id":"6a0e7c1a164dbbc68a26c57a","name":"Yang Yang","hidden":false},{"_id":"6a0e7c1a164dbbc68a26c57b","name":"Yu Cheng","hidden":false},{"_id":"6a0e7c1a164dbbc68a26c57c","name":"Yafu Li","hidden":false}],"publishedAt":"2026-05-19T00:00:00.000Z","submittedOnDailyAt":"2026-05-22T00:00:00.000Z","title":"π-Bench: Evaluating Proactive Personal Assistant Agents in Long-Horizon Workflows","submittedOnDailyBy":{"_id":"689ec537196ab997b13dc977","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/689ec537196ab997b13dc977/yXA_pd8ndjBIIg1Hx59QJ.png","isPro":false,"fullname":"Haoran Zhang","user":"zzzhr97","type":"user","name":"zzzhr97"},"summary":"The rise of personal assistant agents, e.g., OpenClaw, highlights the growing potential of large language models to support users across everyday life and work. A core challenge in these settings is proactive assistance, since users often begin with underspecified requests and leave important needs, constraints, or preferences unstated. However, existing benchmarks rarely evaluate whether agents can identify and act on such hidden intents before they are explicitly stated, especially in sustained multi-turn interactions where user needs emerge gradually. To address this gap, we introduce π-Bench, a benchmark for proactive assistance comprising 100 multi-turn tasks across 5 domain-specific user personas. By incorporating hidden user intents, inter-task dependencies, and cross-session continuity, π-Bench evaluates agents' ability to anticipate and address user needs over extended interactions, jointly measuring proactivity and task completion in long-horizon trajectories that better reflect real-world use. Experiments show (1) proactive assistance remains challenging, (2) a clear distinction between task completion and proactivity, and (3) the value of prior interaction for proactive intent resolution in later tasks.","upvotes":73,"discussionId":"6a0e7c1a164dbbc68a26c57d","projectPage":"https://simplified-reasoning.github.io/Pi-Bench","githubRepo":"https://github.com/Simplified-Reasoning/Pi-Bench","githubRepoAddedBy":"user","ai_summary":"Proactive assistance in personal agent systems requires identifying hidden user intents through sustained multi-turn interactions, which current benchmarks fail to adequately evaluate.","ai_keywords":["personal assistant agents","large language models","proactive assistance","user intents","multi-turn interactions","domain-specific user personas","long-horizon trajectories","task completion","proactivity"],"githubStars":7,"organization":{"_id":"6a03a74a34f1cffc0570e62d","name":"Simplified-Reasoning","fullname":"Simplified Reasoning","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/62495cb96ee7ee6b646db130/S-AqbmitJFm2PxXyZrX7H.jpeg"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"689ec537196ab997b13dc977","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/689ec537196ab997b13dc977/yXA_pd8ndjBIIg1Hx59QJ.png","isPro":false,"fullname":"Haoran Zhang","user":"zzzhr97","type":"user"},{"_id":"64cb54da1af278541d663708","avatarUrl":"/avatars/c44507cc92bb2e83154bad31b90ce6dd.svg","isPro":false,"fullname":"Xiaoye Qu","user":"Xiaoye08","type":"user"},{"_id":"65697feb9fb2d79a79e14e0a","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65697feb9fb2d79a79e14e0a/wVGaBjn8pQIJneZWSFIwS.jpeg","isPro":false,"fullname":"lei","user":"bingyang-lei","type":"user"},{"_id":"67247adb73d1eb17b6bfd27c","avatarUrl":"/avatars/57bdbb7362f9854c87dd0a71ae071652.svg","isPro":false,"fullname":"Zefeng He","user":"yhx12","type":"user"},{"_id":"63f3502a520c14618925825a","avatarUrl":"/avatars/e986a2a6625e7be6890616a417f908d2.svg","isPro":false,"fullname":"Yafu Li","user":"yaful","type":"user"},{"_id":"6418228b83957c4eaaad4d01","avatarUrl":"/avatars/b6af01d09bba5d5ba7bf4a62914ca468.svg","isPro":false,"fullname":"wang","user":"astrid01052","type":"user"},{"_id":"6785e04ce63a669874f166e9","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6785e04ce63a669874f166e9/xSUEAmRm-KadDNOYiOZew.jpeg","isPro":false,"fullname":"dodojorid","user":"yizhuoli","type":"user"},{"_id":"620f5a1c3f76c50e6458a9b6","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620f5a1c3f76c50e6458a9b6/pXh_f5F0UvufxuUa-eS-v.jpeg","isPro":true,"fullname":"Peiyu Wang","user":"OrlandoHugBot","type":"user"},{"_id":"682fe5e1c8ddcbd4645ec29d","avatarUrl":"/avatars/843a5c6ba36230c02cce4a8bf9d47319.svg","isPro":false,"fullname":"Mehmet Yılmaz","user":"mY1lmaz","type":"user"},{"_id":"682fe2641af275ff5c4a06e0","avatarUrl":"/avatars/f939d34170c74fc873a2abb6e97680ff.svg","isPro":false,"fullname":"Emily Carter","user":"EmilyC11","type":"user"},{"_id":"68075d12dadb28dddc14a509","avatarUrl":"/avatars/7a9d86c6fb56ece2ed5bf36350142612.svg","isPro":false,"fullname":"Finch","user":"KKKepler","type":"user"},{"_id":"652e39fea3da41257d6ec253","avatarUrl":"/avatars/fc00bd8bf3704afb269b45b5bf7fd1b9.svg","isPro":false,"fullname":"Li Luo","user":"ReyLock","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":2,"organization":{"_id":"6a03a74a34f1cffc0570e62d","name":"Simplified-Reasoning","fullname":"Simplified Reasoning","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/62495cb96ee7ee6b646db130/S-AqbmitJFm2PxXyZrX7H.jpeg"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2605/2605.14678.md"}">
π-Bench: Evaluating Proactive Personal Assistant Agents in Long-Horizon Workflows
Authors: ,
,
,
,
,
,
,
,
,
,
,
,
Abstract
Proactive assistance in personal agent systems requires identifying hidden user intents through sustained multi-turn interactions, which current benchmarks fail to adequately evaluate.
AI-generated summary
The rise of personal assistant agents, e.g., OpenClaw, highlights the growing potential of large language models to support users across everyday life and work. A core challenge in these settings is proactive assistance, since users often begin with underspecified requests and leave important needs, constraints, or preferences unstated. However, existing benchmarks rarely evaluate whether agents can identify and act on such hidden intents before they are explicitly stated, especially in sustained multi-turn interactions where user needs emerge gradually. To address this gap, we introduce π-Bench, a benchmark for proactive assistance comprising 100 multi-turn tasks across 5 domain-specific user personas. By incorporating hidden user intents, inter-task dependencies, and cross-session continuity, π-Bench evaluates agents' ability to anticipate and address user needs over extended interactions, jointly measuring proactivity and task completion in long-horizon trajectories that better reflect real-world use. Experiments show (1) proactive assistance remains challenging, (2) a clear distinction between task completion and proactivity, and (3) the value of prior interaction for proactive intent resolution in later tasks.
Community
The rise of personal assistant agents, e.g., OpenClaw, highlights the growing potential of large language models to support users across everyday life and work. A core challenge in these settings is proactive assistance, since users often begin with underspecified requests and leave important needs, constraints, or preferences unstated. However, existing benchmarks rarely evaluate whether agents can identify and act on such hidden intents before they are explicitly stated, especially in sustained multi-turn interactions where user needs emerge gradually. To address this gap, we introduce π-Bench, a benchmark for proactive assistance comprising 100 multi-turn tasks across 5 domain-specific user personas. By incorporating hidden user intents, inter-task dependencies, and cross-session continuity, π-Bench evaluates agents’ ability to anticipate and address user needs over extended interactions, jointly measuring proactivity and task completion in long-horizon trajectories that better reflect real-world use. Experiments show (1) proactive assistance remains challenging, (2) a clear distinction between task completion and proactivity, and (3) the value of prior interaction for proactive intent resolution in later tasks.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2605.14678 in a model README.md to link it from this page.
Cite arxiv.org/abs/2605.14678 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2605.14678 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.