Accepted at ASE'26</p>\n","updatedAt":"2026-08-05T03:37:09.607Z","author":{"_id":"61bd6bbb92dd56338582ebbd","avatarUrl":"/avatars/3637ab04f9342cf15340a47c9645722a.svg","fullname":"Imam Nur Bani Yusuf","name":"imamnurby","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8596805334091187},"editors":["imamnurby"],"editorAvatarUrls":["/avatars/3637ab04f9342cf15340a47c9645722a.svg"],"reactions":[],"isReport":false}},{"id":"6a72b0a05fdcf010ad34dd60","author":{"_id":"61bd6bbb92dd56338582ebbd","avatarUrl":"/avatars/3637ab04f9342cf15340a47c9645722a.svg","fullname":"Imam Nur Bani Yusuf","name":"imamnurby","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false},"createdAt":"2026-08-05T03:40:16.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Blog post: https://explainbench.github.io/blog/introducing-explainbench/","html":"<p>Blog post: <a href=\"https://explainbench.github.io/blog/introducing-explainbench/\" rel=\"nofollow\">https://explainbench.github.io/blog/introducing-explainbench/</a></p>\n","updatedAt":"2026-08-05T03:40:16.289Z","author":{"_id":"61bd6bbb92dd56338582ebbd","avatarUrl":"/avatars/3637ab04f9342cf15340a47c9645722a.svg","fullname":"Imam Nur Bani Yusuf","name":"imamnurby","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.29119572043418884},"editors":["imamnurby"],"editorAvatarUrls":["/avatars/3637ab04f9342cf15340a47c9645722a.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2607.26451","authors":[{"_id":"6a72afac1a375f948521c3fc","name":"Zhiyuan Pan","hidden":false},{"_id":"6a72afac1a375f948521c3fd","name":"Sungmin Kang","hidden":false},{"_id":"6a72afac1a375f948521c3fe","name":"Imam Nur Bani Yusuf","hidden":false},{"_id":"6a72afac1a375f948521c3ff","name":"Abhik Roychoudhury","hidden":false}],"publishedAt":"2026-07-29T00:00:00.000Z","submittedOnDailyAt":"2026-08-05T00:00:00.000Z","title":"ExplainBench: Evaluating Code Explanations from Agents","submittedOnDailyBy":{"_id":"61bd6bbb92dd56338582ebbd","avatarUrl":"/avatars/3637ab04f9342cf15340a47c9645722a.svg","isPro":false,"fullname":"Imam Nur Bani Yusuf","user":"imamnurby","type":"user","name":"imamnurby"},"summary":"Large Language Model (LLM) agents have seen rapid adoption in software engineering. As agents take a greater role in the actual generation of code, they are making larger changes, spanning tens to hundreds of lines. This makes manual review of agent results increasingly infeasible, leading developers to turn to explanations to understand enacted changes. Despite this, there are no benchmarks that evaluate the trustworthiness of agent-generated explanations. To bridge this gap, we propose ExplainBench, a benchmark to automatically evaluate explanations from coding agents. ExplainBench is based on the intuition that informative explanations should enable an LLM to correctly answer questions, allowing quantitative comparison of explanation quality between agents. With this observation, we construct a suite of questions that evaluates whether explanations accurately describe (1) the intended behavior of buggy code and (2) the effect of applying the agent patch itself. Experiments first reveal that explanation quality is a distinct axis of agent evaluation: ExplainBench ranks agents differently from the widely-used SWE-bench Verified benchmark. A deeper breakdown of explanation quality in agents shows frequent problems in explanations, such that explanations often claim that a patch is correct when it is not. Based on this insight, we implement and evaluate an explanation audit agent which runs additional tests to validate and refine explanations. This agent improved the explanations of all evaluated agents, demonstrating agent explanations can be automatically made more trustworthy.","upvotes":4,"discussionId":"6a72afac1a375f948521c400","projectPage":"https://explainbench.github.io","githubRepo":"https://github.com/explainbench/explainbench-cli","githubRepoAddedBy":"user","githubStars":0},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"66d8512c54209e9101811e8e","avatarUrl":"/avatars/62dfd8e6261108f2508efe678d5a2a57.svg","isPro":false,"fullname":"M Saad Salman","user":"MSS444","type":"user"},{"_id":"61bd6bbb92dd56338582ebbd","avatarUrl":"/avatars/3637ab04f9342cf15340a47c9645722a.svg","isPro":false,"fullname":"Imam Nur Bani Yusuf","user":"imamnurby","type":"user"},{"_id":"64408e05518271b0d1bfe163","avatarUrl":"/avatars/a5e018d7517f5dcf5c93ef5550a7a190.svg","isPro":false,"fullname":"Sungmin Kang","user":"smkang96","type":"user"},{"_id":"63ac5701c21e60a3e9b58aa7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63ac5701c21e60a3e9b58aa7/g6EX7diOpuA94R2ab-rZC.png","isPro":true,"fullname":"Dipankar Sarkar","user":"dipankarsarkar","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2607/2607.26451.md","query":{}}">

Get this paper in your agent:

Don't have the latest CLI?

Cite arxiv.org/abs/2607.26451 in a model README.md to link it from this page.

Cite arxiv.org/abs/2607.26451 in a dataset README.md to link it from this page.

Cite arxiv.org/abs/2607.26451 in a Space README.md to link it from this page.

Add this paper to a collection to link it from this page.