This is an automated message from the <a href=\"https://huggingface.co/librarian-bots\">Librarian Bot</a>. I found the following papers similar to this paper. </p>\n<p>The following papers were recommended by the Semantic Scholar API </p>\n<ul>\n<li><a href=\"https://huggingface.co/papers/2608.11669\">Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2609.01245\">Explore More, Drift Less: Outcome-Only Reinforcement Learning Can Suffice for Long-Horizon Interactive Agents</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2607.13988\">TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2607.23263\">SeekJudge: A Practical Reward Framework for Reinforcement Learning in Computer-Use Agents</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2608.18852\">SkillGate: Training In-Policy Skill Selection in Long-Horizon Agents</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2608.28128\">VICT: Verifier-Instrumented Credit Tracing for Long-Horizon LLM Agent Reinforcement Learning</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2607.11172\">STAMP: Provenance-Guided Credit Assignment for Deep Search Agents</a> (2026)</li>\n</ul>\n<p> Please give a thumbs up to this comment if you found it helpful!</p>\n<p> If you want recommendations for any Paper on Hugging Face checkout <a href=\"https://huggingface.co/spaces/librarian-bots/recommend_similar_papers\">this</a> Space</p>\n<p> You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: <code>@librarian-bot recommend</code></p>\n","updatedAt":"2026-09-05T01:23:00.197Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":379,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7361451983451843},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}},{"id":"6a9c06ea523b03f8e8df1427","author":{"_id":"658412f93a84a40185adaf37","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/658412f93a84a40185adaf37/FKXH7e1jj09KO1v-B5sER.jpeg","fullname":"Aamer Mihaysi","name":"O96a","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false},"createdAt":"2026-09-05T12:11:22.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Honest question on the rubric generator: if it's learned from the policy's own behavior, what stops it from grading what the policy already does well? That's a self-fulfilling loop — the reward function adapts to the agent instead of the task. Maybe the dynamic part is meant to break that, but I'd want to see the rubric drift over training. And I'm skeptical per-step credit assignment survives contact with a real browser. WebArena would be the test — has anyone run this there, or is it still mostly scripted environments?","html":"<p>Honest question on the rubric generator: if it's learned from the policy's own behavior, what stops it from grading what the policy already does well? That's a self-fulfilling loop — the reward function adapts to the agent instead of the task. Maybe the dynamic part is meant to break that, but I'd want to see the rubric drift over training. And I'm skeptical per-step credit assignment survives contact with a real browser. WebArena would be the test — has anyone run this there, or is it still mostly scripted environments?</p>\n","updatedAt":"2026-09-05T12:11:22.925Z","author":{"_id":"658412f93a84a40185adaf37","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/658412f93a84a40185adaf37/FKXH7e1jj09KO1v-B5sER.jpeg","fullname":"Aamer Mihaysi","name":"O96a","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9584542512893677},"editors":["O96a"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/658412f93a84a40185adaf37/FKXH7e1jj09KO1v-B5sER.jpeg"],"reactions":[],"isReport":false},"replies":[{"id":"6a9c1b47c296a85df73b705e","author":{"_id":"64ae4890b575c5e27226cef2","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64ae4890b575c5e27226cef2/iKyTZoIa8Msy_jNKgHJn1.jpeg","fullname":"Shubham Gandhi","name":"shubhamrgandhi","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false},"createdAt":"2026-09-05T13:38:15.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Great question! We explicitly prompt the dynamic rubric generator to propose criteria based on where the agent falls short. After merging criteria across the rollout group, we drop any criterion that all rollouts pass unanimously so that already mastered behavior does not dilute the reward signal. Figure 4 shows the effect of doing this - static rubrics quickly saturate, whereas dynamic rubrics remain discriminative as the policy improves. The final AppWorld gains are also measured using the official verifier, which is never used to construct the training reward. We have not trained / tested DRACO on WebArena. We agree that real browser interaction would be a useful next test. The code is open source, and we would be glad to see it extended to that setting! - https://github.com/IBM/draco","html":"<p>Great question! We explicitly prompt the dynamic rubric generator to propose criteria based on where the agent falls short. After merging criteria across the rollout group, we drop any criterion that all rollouts pass unanimously so that already mastered behavior does not dilute the reward signal. Figure 4 shows the effect of doing this - static rubrics quickly saturate, whereas dynamic rubrics remain discriminative as the policy improves. The final AppWorld gains are also measured using the official verifier, which is never used to construct the training reward. We have not trained / tested DRACO on WebArena. We agree that real browser interaction would be a useful next test. The code is open source, and we would be glad to see it extended to that setting! - <a href=\"https://github.com/IBM/draco\" rel=\"nofollow\">https://github.com/IBM/draco</a></p>\n","updatedAt":"2026-09-05T13:38:15.823Z","author":{"_id":"64ae4890b575c5e27226cef2","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64ae4890b575c5e27226cef2/iKyTZoIa8Msy_jNKgHJn1.jpeg","fullname":"Shubham Gandhi","name":"shubhamrgandhi","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9269978404045105},"editors":["shubhamrgandhi"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/64ae4890b575c5e27226cef2/iKyTZoIa8Msy_jNKgHJn1.jpeg"],"reactions":[],"isReport":false,"parentCommentId":"6a9c06ea523b03f8e8df1427"}}]},{"id":"6a9c1aefd9a445dcb931b1f4","author":{"_id":"64ae4890b575c5e27226cef2","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64ae4890b575c5e27226cef2/iKyTZoIa8Msy_jNKgHJn1.jpeg","fullname":"Shubham Gandhi","name":"shubhamrgandhi","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false},"createdAt":"2026-09-05T13:36:47.000Z","type":"comment","data":{"edited":true,"hidden":true,"hiddenBy":"","hiddenReason":"Resolved","latest":{"raw":"This comment has been hidden","html":"This comment has been hidden","updatedAt":"2026-09-05T13:38:05.544Z","author":{"_id":"64ae4890b575c5e27226cef2","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64ae4890b575c5e27226cef2/iKyTZoIa8Msy_jNKgHJn1.jpeg","fullname":"Shubham Gandhi","name":"shubhamrgandhi","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"editors":[],"editorAvatarUrls":[],"reactions":[]}}],"primaryEmailConfirmed":false,"paper":{"id":"2609.04094","authors":[{"_id":"6a9a3fd08f7c3b75572394e8","name":"Shubham Gandhi","hidden":false},{"_id":"6a9a3fd08f7c3b75572394e9","name":"Saurabh Goyal","hidden":false},{"_id":"6a9a3fd08f7c3b75572394ea","name":"Kiran Kate","hidden":false},{"_id":"6a9a3fd08f7c3b75572394eb","name":"Yara Rizk","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/64ae4890b575c5e27226cef2/6RtZWU-LVBoNYLl7Id_uy.png","https://cdn-uploads.huggingface.co/production/uploads/64ae4890b575c5e27226cef2/O8XCeNAEa9uvCz6SKyy2L.png","https://cdn-uploads.huggingface.co/production/uploads/64ae4890b575c5e27226cef2/dOLP93DjHdPS_GYwYnWqK.png","https://cdn-uploads.huggingface.co/production/uploads/64ae4890b575c5e27226cef2/bcdjCiMKek160Q9GbWOhB.png","https://cdn-uploads.huggingface.co/production/uploads/64ae4890b575c5e27226cef2/aXYbsGejA3YBorM6V1pYV.png","https://cdn-uploads.huggingface.co/production/uploads/64ae4890b575c5e27226cef2/bjKYFVebUJM77ZpeCHx0R.png","https://cdn-uploads.huggingface.co/production/uploads/64ae4890b575c5e27226cef2/5nG6VGTyZuAT8mFQtum4E.png"],"publishedAt":"2026-09-03T00:00:00.000Z","submittedOnDailyAt":"2026-09-04T00:00:00.000Z","title":"DRACO: Fine-Grained Credit Assignment with Dynamic Rubrics for Long-Horizon Agent Training","submittedOnDailyBy":{"_id":"64ae4890b575c5e27226cef2","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64ae4890b575c5e27226cef2/iKyTZoIa8Msy_jNKgHJn1.jpeg","isPro":false,"fullname":"Shubham Gandhi","user":"shubhamrgandhi","type":"user","name":"shubhamrgandhi"},"summary":"Reinforcement Learning from Verifiable Rewards works well when a task has a programmatic checker, but most long-horizon agent domains have none. We work in the outcome-blind setting, where ground-truth success signals are not available. Multi-criteria rubrics are a popular way to supply such a reward; they are scored once per trajectory, but a single scalar is a poor signal across tens of steps. We propose DRACO: Distributing Rubric-based Advantage for Credit Optimization. It generates rubrics dynamically during training to track the policy's evolving capability, scores those rubrics once per completed trajectory, and redistributes that judgment over the steps responsible for annotated rubrics to produce differentiated per-step advantages in GRPO. The redistribution is closed-form and does not introduce any trained attribution module. On AppWorld, DRACO gains 15.9 points over the base model and 5.3 points over GRPO trained with a sparse ground-truth reward, despite not using any verifiers itself. On out-of-domain Tau-Bench, it gains 5.3 points over the base model even without a frontier judge, beating both ground-truth-reward training and other rubric-based training settings. The code for DRACO is available at https://github.com/IBM/draco.","upvotes":23,"discussionId":"6a9a3fd18f7c3b75572394ec","githubRepo":"https://github.com/IBM/draco","githubRepoAddedBy":"user","ai_summary":"DRACO dynamically generates rubrics and redistributes trajectory-level scores into per-step advantages for reinforcement learning without verifiers, improving long-horizon agent performance.","ai_keywords":["Reinforcement Learning from Verifiable Rewards","multi-criteria rubrics","DRACO","rubric-based advantage","credit optimization","GRPO","per-step advantages","outcome-blind"],"ai_summary_model":"thinkingmachines/Inkling-Small","githubStars":2,"organization":{"_id":"6760ab6c5c9a8ea8370ab95b","name":"ibm-research","fullname":"IBM Research","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/637bfdf60dc13843b468ac20/npxapKcW-cXX3J2JBl2vY.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"64ae4890b575c5e27226cef2","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64ae4890b575c5e27226cef2/iKyTZoIa8Msy_jNKgHJn1.jpeg","isPro":false,"fullname":"Shubham Gandhi","user":"shubhamrgandhi","type":"user"},{"_id":"63ac5701c21e60a3e9b58aa7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63ac5701c21e60a3e9b58aa7/g6EX7diOpuA94R2ab-rZC.png","isPro":true,"fullname":"Dipankar Sarkar","user":"dipankarsarkar","type":"user"},{"_id":"6335298275bed9932474ea77","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6335298275bed9932474ea77/yGLUoF77ATne3z1bu7uJ2.jpeg","isPro":false,"fullname":"Atharva Naik","user":"atharva-naik-1","type":"user"},{"_id":"648749094dea003c6dae810f","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/648749094dea003c6dae810f/gHUHSBt1zrt8wjO1YwTNu.jpeg","isPro":false,"fullname":"Shrey Pandit","user":"SP2001","type":"user"},{"_id":"67eaa79c932c2301b8b37963","avatarUrl":"/avatars/a8e6ed70291c1a54635990fe421f25e1.svg","isPro":false,"fullname":"Saurabh Goyal","user":"saurabhiit2011","type":"user"},{"_id":"667504e68a4064c02bf416e2","avatarUrl":"/avatars/6a745d5c326cb7236f6b42a47dc3baac.svg","isPro":false,"fullname":"Shreyas Bhat","user":"ShreyasB1729","type":"user"},{"_id":"606c82423a0f1f4b6c4358f9","avatarUrl":"/avatars/fbde74cb6bf95cd51412962f0eafa824.svg","isPro":false,"fullname":"Kiran Kate","user":"kirankate06","type":"user"},{"_id":"6a2da6c8ca070ee12c6e396c","avatarUrl":"/avatars/0355287dcabaa67dbc7f0b10b87451f9.svg","isPro":false,"fullname":"Joe Mama","user":"JoeMama123123123","type":"user"},{"_id":"603c44e677a2a899efe25828","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1614562504843-noauth.jpeg","isPro":false,"fullname":"Vijay Viswanathan","user":"viswavi","type":"user"},{"_id":"645e4bc1d9187cd3e07535c2","avatarUrl":"/avatars/05ceb02598a78aefeb479f6d68080d7f.svg","isPro":false,"fullname":"Vinod Muthusamy","user":"vinodmut","type":"user"},{"_id":"69c494d93ca58d7b0288709f","avatarUrl":"/avatars/8e5305061fea0e643c0b38ac3194c4a5.svg","isPro":false,"fullname":"Eyal Shnarch","user":"eishna","type":"user"},{"_id":"649d9ad1500fd8d51a675a93","avatarUrl":"/avatars/c2670566c5cbdfccf87954a9caf7d381.svg","isPro":false,"fullname":"Anupama Murthi","user":"anupamamurthi","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"6760ab6c5c9a8ea8370ab95b","name":"ibm-research","fullname":"IBM Research","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/637bfdf60dc13843b468ac20/npxapKcW-cXX3J2JBl2vY.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2609/2609.04094.md","query":{}}">