HealthAgentBench evaluates frontier agents on realistic healthcare workflows spanning seven task categories with distinct environments and diverse data modalities, including 2D chest X-rays, 3D CT volumes, gigapixel whole-slide pathology images, free-text clinical documents, and structured EHR data. Each task is executed in a terminal environment using real clinical artefacts with minimal instructions, and evaluated against hidden gold labels to determine task success.</p>\n","updatedAt":"2026-07-02T19:49:47.289Z","author":{"_id":"617e7729129c9e67703ffe61","avatarUrl":"/avatars/f47ee9f2f0e2b1075bebf3682ee2f817.svg","fullname":"qianchu liu","name":"qianchu","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.854180634021759},"editors":["qianchu"],"editorAvatarUrls":["/avatars/f47ee9f2f0e2b1075bebf3682ee2f817.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.31179","authors":[{"_id":"6a469cc46ee372f6920de147","name":"Qianchu Liu","hidden":false},{"_id":"6a469cc46ee372f6920de148","name":"Sheng Zhang","hidden":false},{"_id":"6a469cc46ee372f6920de149","name":"Guanghui Qin","hidden":false},{"_id":"6a469cc46ee372f6920de14a","name":"Jeya Maria Jose Valanarasu","hidden":false},{"_id":"6a469cc46ee372f6920de14b","name":"Maximilian Rokuss","hidden":false},{"_id":"6a469cc46ee372f6920de14c","name":"Mingyu Lu","hidden":false},{"_id":"6a469cc46ee372f6920de14d","name":"Timothy Ossowski","hidden":false},{"_id":"6a469cc46ee372f6920de14e","name":"Juan Manuel Zambrano Chaves","hidden":false},{"_id":"6a469cc46ee372f6920de14f","name":"Cliff Wong","hidden":false},{"_id":"6a469cc46ee372f6920de150","name":"Peniel Argaw","hidden":false},{"_id":"6a469cc46ee372f6920de151","name":"Yashna Hasija","hidden":false},{"_id":"6a469cc46ee372f6920de152","name":"Mu Wei","hidden":false},{"_id":"6a469cc46ee372f6920de153","name":"Wen-wai Yim","hidden":false},{"_id":"6a469cc46ee372f6920de154","name":"Qin Liu","hidden":false},{"_id":"6a469cc46ee372f6920de155","name":"Zilin Jing","hidden":false},{"_id":"6a469cc46ee372f6920de156","name":"Jason Entenmann","hidden":false},{"_id":"6a469cc46ee372f6920de157","name":"Naoto Usuyama","hidden":false},{"_id":"6a469cc46ee372f6920de158","name":"Tristan Naumann","hidden":false},{"_id":"6a469cc46ee372f6920de159","name":"Hoifung Poon","hidden":false}],"publishedAt":"2026-06-30T00:00:00.000Z","submittedOnDailyAt":"2026-07-02T00:00:00.000Z","title":"HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents","submittedOnDailyBy":{"_id":"617e7729129c9e67703ffe61","avatarUrl":"/avatars/f47ee9f2f0e2b1075bebf3682ee2f817.svg","isPro":false,"fullname":"qianchu liu","user":"qianchu","type":"user","name":"qianchu"},"summary":"As AI agents become increasingly capable of complex, long-horizon reasoning, rigorous and holistic evaluation is essential for measuring progress toward real-world healthcare applications. We introduce HealthAgentBench, a suite of 54 agentic healthcare tasks across 7 categories each with its unique environment. The benchmark suite spans diverse workflows throughout the patient journey and a broad range of modalities. Each task is designed to replicate an end-to-end clinical workflow: given minimal instructions, an agent must explore raw healthcare data, operate within a complex environment, and execute multi-step solutions that go beyond naive prompting. A final task success rate is reported to provide a single, interpretable metric for HealthAgentBench overall performance for each agent. Evaluating frontier agents on HealthAgentBench, we find that overall task success rate remains low, underscoring the difficulty of the suite. The strongest and the most cost effective agent, Codex GPT-5.5, achieves only approximately 42% success rate. Beyond aggregate performance, HealthAgentBench reveals nuanced strengths and weaknesses across task categories. Frontier agents show promise in automatically developing research modeling pipelines over EHR data, but medical imaging remains especially challenging, particularly for Claude Code models, while Codex GPT-5.5 shows emerging capability. Tasks that combine large search spaces with compositional reasoning requirements remain difficult for all current agents. Together, these results suggest that HealthAgentBench provides a challenging and realistic benchmark with substantial room for future progress. We release our benchmark at https://github.com/microsoft/HealthAgentBench.","upvotes":2,"discussionId":"6a469cc46ee372f6920de15a","projectPage":"https://microsoft.github.io/HealthAgentBench/","githubRepo":"https://github.com/microsoft/HealthAgentBench","githubRepoAddedBy":"user","ai_summary":"HealthAgentBench presents a comprehensive evaluation framework with 54 healthcare tasks across 7 categories to assess AI agents' capabilities in complex clinical workflows, revealing significant challenges in medical imaging and compositional reasoning while showing promise in EHR data analysis.","ai_keywords":["agentic healthcare tasks","clinical workflows","EHR data","medical imaging","compositional reasoning","task success rate","frontier agents","Codex GPT-5.5","Claude Code models"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","githubStars":8,"organization":{"_id":"5e6485f787403103f9f1055e","name":"microsoft","fullname":"Microsoft","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1583646260758-5e64858c87403103f9f1055d.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"617e7729129c9e67703ffe61","avatarUrl":"/avatars/f47ee9f2f0e2b1075bebf3682ee2f817.svg","isPro":false,"fullname":"qianchu liu","user":"qianchu","type":"user"},{"_id":"6a2da6c8ca070ee12c6e396c","avatarUrl":"/avatars/0355287dcabaa67dbc7f0b10b87451f9.svg","isPro":false,"fullname":"Joe Mama","user":"JoeMama123123123","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"5e6485f787403103f9f1055e","name":"microsoft","fullname":"Microsoft","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1583646260758-5e64858c87403103f9f1055d.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.31179.md","query":{}}">

Get this paper in your agent:

Don't have the latest CLI?

Cite arxiv.org/abs/2606.31179 in a model README.md to link it from this page.

Cite arxiv.org/abs/2606.31179 in a dataset README.md to link it from this page.

Cite arxiv.org/abs/2606.31179 in a Space README.md to link it from this page.

Add this paper to a collection to link it from this page.