Recent multimodal large language models have shown great promise in clinical image reasoning, but existing post-training pipelines remain predominantly outcome-centric, relying on final answer correctness or sequence-level preferences. This suffers from sparse credit assignment, making it difficult to optimize the reasoning process essential for clinical applications. Our analysis reveals that cascading errors from early-stage reasoning failures are a leading cause of incorrect predictions in medical visual question answering (VQA) benchmarks. Motivated by this, we propose Medical Reasoning-aware Policy Optimization (MRPO), an RL algorithm that incorporates step-wise process rewards. When the final answer is incorrect, MRPO assigns exponentially larger penalties to tokens in earlier invalid reasoning steps, breaking failure cascades without compromising successful paths. Across three multimodal LLM backbones, MRPO consistently outperforms standard GRPO and a recent RL baseline, and on Qwen3-VL-8B-Instruct even surpasses substantially larger medical MLLMs such as HuatuoGPT-Vision-34B by 2.79 points. Moreover, MRPO reduces early-stage reasoning failures from 64.0% to 13.0%, showing that targeted mitigation of cascading failures improves both reasoning quality and final answer accuracy. Our code is available at this https URL (<a href=\"https://github.com/dmis-lab/MRPO\" rel=\"nofollow\">https://github.com/dmis-lab/MRPO</a>)</p>\n<p>Models: <a href=\"https://huggingface.co/collections/dmis-lab/mrpo\">https://huggingface.co/collections/dmis-lab/mrpo</a></p>\n","updatedAt":"2026-07-03T06:27:52.375Z","author":{"_id":"64587be872b60ae7a3817858","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64587be872b60ae7a3817858/BbdOOxOCEzWTvEpkWp8MM.png","fullname":"Minbyul Jeong","name":"Minbyul","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":5,"isUserFollowing":false}},"numEdits":1,"identifiedLanguage":{"language":"en","probability":0.900011420249939},"editors":["Minbyul"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/64587be872b60ae7a3817858/BbdOOxOCEzWTvEpkWp8MM.png"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.31825","authors":[{"_id":"6a471ff96ee372f6920de2e7","name":"Junha Jung","hidden":false},{"_id":"6a471ff96ee372f6920de2e8","name":"Minbyul Jeong","hidden":false},{"_id":"6a471ff96ee372f6920de2e9","name":"Suhyeon Lim","hidden":false},{"_id":"6a471ff96ee372f6920de2ea","name":"Sungwook Jung","hidden":false},{"_id":"6a471ff96ee372f6920de2eb","name":"Jaehoon Yun","hidden":false},{"_id":"6a471ff96ee372f6920de2ec","name":"Taeyun Roh","hidden":false},{"_id":"6a471ff96ee372f6920de2ed","name":"Mujeen Sung","hidden":false},{"_id":"6a471ff96ee372f6920de2ee","name":"Jaewoo Kang","hidden":false}],"publishedAt":"2026-06-30T00:00:00.000Z","submittedOnDailyAt":"2026-07-03T00:00:00.000Z","title":"Breaking Failure Cascades: Step-Aware Reinforcement Learning for Medical Multimodal Reasoning","submittedOnDailyBy":{"_id":"64587be872b60ae7a3817858","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64587be872b60ae7a3817858/BbdOOxOCEzWTvEpkWp8MM.png","isPro":false,"fullname":"Minbyul Jeong","user":"Minbyul","type":"user","name":"Minbyul"},"summary":"Recent multimodal large language models have shown great promise in clinical image reasoning, but existing post-training pipelines remain predominantly outcome-centric, relying on final answer correctness or sequence-level preferences. This suffers from sparse credit assignment, making it difficult to optimize the reasoning process essential for clinical applications. Our analysis reveals that cascading errors from early-stage reasoning failures are a leading cause of incorrect predictions in medical visual question answering (VQA) benchmarks. Motivated by this, we propose Medical Reasoning-aware Policy Optimization (MRPO), an RL algorithm that incorporates step-wise process rewards. When the final answer is incorrect, MRPO assigns exponentially larger penalties to tokens in earlier invalid reasoning steps, breaking failure cascades without compromising successful paths. Across three multimodal LLM backbones, MRPO consistently outperforms standard GRPO and a recent RL baseline, and on Qwen3-VL-8B-Instruct even surpasses substantially larger medical MLLMs such as HuatuoGPT-Vision-34B by 2.79 points. Moreover, MRPO reduces early-stage reasoning failures from 64.0% to 13.0%, showing that targeted mitigation of cascading failures improves both reasoning quality and final answer accuracy. Our code is available at https://github.com/dmis-lab/MRPO","upvotes":10,"discussionId":"6a471ffa6ee372f6920de2ef","projectPage":"https://jungjunha.github.io/MRPO-page/","githubRepo":"https://github.com/dmis-lab/MRPO","githubRepoAddedBy":"user","ai_summary":"A reinforcement learning approach called MRPO is introduced to improve clinical image reasoning by addressing cascading errors through step-wise process rewards, demonstrating superior performance over existing methods.","ai_keywords":["multimodal large language models","medical visual question answering","reinforcement learning","policy optimization","cascading errors","step-wise process rewards","reasoning process","final answer correctness","credit assignment"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","githubStars":2},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"67bc90b961b6284817237bda","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/ucJr5q-ub4RDkswy2Kw9Y.png","isPro":false,"fullname":"Junha Jung","user":"JunhaJung","type":"user"},{"_id":"64587be872b60ae7a3817858","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64587be872b60ae7a3817858/BbdOOxOCEzWTvEpkWp8MM.png","isPro":false,"fullname":"Minbyul Jeong","user":"Minbyul","type":"user"},{"_id":"63f1de31f4e30ffd2bcd626b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63f1de31f4e30ffd2bcd626b/aPHgcUj0NN68_fIEym3HE.jpeg","isPro":false,"fullname":"Suhyeong Park","user":"Codingchild","type":"user"},{"_id":"664b233490135abe9bbbb72a","avatarUrl":"/avatars/d2175e2e2956ae1b61093cd424bc903e.svg","isPro":false,"fullname":"Hoonick Lee","user":"hoonick","type":"user"},{"_id":"64bde788140491ca9f649111","avatarUrl":"/avatars/6157c3dec3d01f8970b2a5835ccd2b66.svg","isPro":false,"fullname":"sungwook Jung","user":"oceanstar","type":"user"},{"_id":"66d914d4c39b38d37fd46989","avatarUrl":"/avatars/8acf2b96957efe5c23119f97c54d06eb.svg","isPro":false,"fullname":"Dongyoung Lee","user":"GBEdge","type":"user"},{"_id":"69f0644c31b9968683d0a6b4","avatarUrl":"/avatars/2c0abc3b74d1d98c2e7765239131c7f0.svg","isPro":false,"fullname":"Hyeon Hwang","user":"hyeon-trl","type":"user"},{"_id":"689431c991b338f15ee89b51","avatarUrl":"/avatars/f3d19eb7dc9dc086c4758e3fe778121d.svg","isPro":false,"fullname":"sanghoon lee","user":"sanghoonAIGEN","type":"user"},{"_id":"69a3ef8c15c3e29635ecc88b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/TrMlSXieB0pSMbtdfQ1eJ.jpeg","isPro":false,"fullname":"Victoria Rodriguez","user":"rileyrodriguez9","type":"user"},{"_id":"5efbdc4ac3896117eab961a9","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1602668910270-5efbdc4ac3896117eab961a9.png","isPro":false,"fullname":"Data Mining and Information Systems Lab","user":"dmis-lab","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.31825.md","query":{}}">
Breaking Failure Cascades: Step-Aware Reinforcement Learning for Medical Multimodal Reasoning
Abstract
A reinforcement learning approach called MRPO is introduced to improve clinical image reasoning by addressing cascading errors through step-wise process rewards, demonstrating superior performance over existing methods.
Recent multimodal large language models have shown great promise in clinical image reasoning, but existing post-training pipelines remain predominantly outcome-centric, relying on final answer correctness or sequence-level preferences. This suffers from sparse credit assignment, making it difficult to optimize the reasoning process essential for clinical applications. Our analysis reveals that cascading errors from early-stage reasoning failures are a leading cause of incorrect predictions in medical visual question answering (VQA) benchmarks. Motivated by this, we propose Medical Reasoning-aware Policy Optimization (MRPO), an RL algorithm that incorporates step-wise process rewards. When the final answer is incorrect, MRPO assigns exponentially larger penalties to tokens in earlier invalid reasoning steps, breaking failure cascades without compromising successful paths. Across three multimodal LLM backbones, MRPO consistently outperforms standard GRPO and a recent RL baseline, and on Qwen3-VL-8B-Instruct even surpasses substantially larger medical MLLMs such as HuatuoGPT-Vision-34B by 2.79 points. Moreover, MRPO reduces early-stage reasoning failures from 64.0% to 13.0%, showing that targeted mitigation of cascading failures improves both reasoning quality and final answer accuracy. Our code is available at https://github.com/dmis-lab/MRPO
Community
Recent multimodal large language models have shown great promise in clinical image reasoning, but existing post-training pipelines remain predominantly outcome-centric, relying on final answer correctness or sequence-level preferences. This suffers from sparse credit assignment, making it difficult to optimize the reasoning process essential for clinical applications. Our analysis reveals that cascading errors from early-stage reasoning failures are a leading cause of incorrect predictions in medical visual question answering (VQA) benchmarks. Motivated by this, we propose Medical Reasoning-aware Policy Optimization (MRPO), an RL algorithm that incorporates step-wise process rewards. When the final answer is incorrect, MRPO assigns exponentially larger penalties to tokens in earlier invalid reasoning steps, breaking failure cascades without compromising successful paths. Across three multimodal LLM backbones, MRPO consistently outperforms standard GRPO and a recent RL baseline, and on Qwen3-VL-8B-Instruct even surpasses substantially larger medical MLLMs such as HuatuoGPT-Vision-34B by 2.79 points. Moreover, MRPO reduces early-stage reasoning failures from 64.0% to 13.0%, showing that targeted mitigation of cascading failures improves both reasoning quality and final answer accuracy. Our code is available at this https URL (https://github.com/dmis-lab/MRPO)
Models: https://huggingface.co/collections/dmis-lab/mrpo
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2606.31825 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2606.31825 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.