We observe that successful MDLM generations exhibit stable confidence dynamics over answer-relevant positions, and that unreliable trajectories can often be corrected using promising intermediate states from other models. Building on this observation, we propose TIE, which enables knowledge fusion across heterogeneous MDLMs by iteratively identifying and relaying reliable decoding trajectories during generation.</p>\n","updatedAt":"2026-06-16T03:10:18.331Z","author":{"_id":"67f778ddbb19958f5d96c2a8","avatarUrl":"/avatars/49a3f119b456ff94f28f09b2fe78bb18.svg","fullname":"Heecheol Yun","name":"yoon6503","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9129940867424011},"editors":["yoon6503"],"editorAvatarUrls":["/avatars/49a3f119b456ff94f28f09b2fe78bb18.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.16281","authors":[{"_id":"6a30b8b3a0d4daae4285fd74","name":"Heecheol Yun","hidden":false},{"_id":"6a30b8b3a0d4daae4285fd75","name":"Joonhyung Park","hidden":false},{"_id":"6a30b8b3a0d4daae4285fd76","user":{"_id":"6666b61eaf95872a03a0a673","avatarUrl":"/avatars/fc0c144cf6307357d45d7ca2d6ba8d2f.svg","isPro":false,"fullname":"Joowon","user":"kjwispro","type":"user","name":"kjwispro"},"name":"Joowon Kim","status":"claimed_verified","statusLastChangedAt":"2026-06-16T12:07:15.743Z","hidden":false},{"_id":"6a30b8b3a0d4daae4285fd77","name":"Eunho Yang","hidden":false}],"publishedAt":"2026-06-15T00:00:00.000Z","submittedOnDailyAt":"2026-06-16T00:00:00.000Z","title":"Who Should Lead Decoding Now? Tracking Reliable Trajectories for Ensembling Masked Diffusion Language Models","submittedOnDailyBy":{"_id":"67f778ddbb19958f5d96c2a8","avatarUrl":"/avatars/49a3f119b456ff94f28f09b2fe78bb18.svg","isPro":false,"fullname":"Heecheol Yun","user":"yoon6503","type":"user","name":"yoon6503"},"summary":"Masked Diffusion Language Models (MDLMs) have emerged as a distinct paradigm for sequence generation. As MDLMs become diverse in capabilities and knowledge coverage, an important question is how to combine their knowledge. Toward this, we first investigate the unique decoding dynamics of MDLMs. We find that successful generations exhibit stable confidence dynamics over answer-relevant positions, while unreliable trajectories can often be corrected by injecting promising intermediate states from other models. Guided by this observation, we propose TIE (Trajectory-based Iterative Ensembling), a knowledge fusion framework in which MDLMs iteratively identify reliable decoding trajectories and relay them across models. TIE tracks confidence dynamics over answer-relevant positions to determine which model currently follows a more reliable trajectory and selectively transfers partially denoised sequences across models. As the model on the more promising trajectory often changes across denoising steps, TIE allows different models to contribute complementary strengths at different stages of generation. Strong performance across diverse reasoning tasks, along with our analyses, suggests that TIE offers a practical approach to the underexplored problem of MDLM ensembling.","upvotes":23,"discussionId":"6a30b8b3a0d4daae4285fd78","ai_summary":"Masked diffusion language models exhibit unique decoding dynamics where reliable trajectories show stable confidence patterns, enabling iterative ensemble methods that transfer partially denoised sequences between models based on confidence evolution.","ai_keywords":["masked diffusion language models","decoding dynamics","confidence dynamics","trajectory-based iterative ensembling","denoising steps","partially denoised sequences","ensemble methods"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","organization":{"_id":"6475760c33192631bad2bb38","name":"kaist-ai","fullname":"KAIST AI","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/6469949654873f0043b09c22/aaZFiyXe1qR-Dmy_xq67m.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"67f778ddbb19958f5d96c2a8","avatarUrl":"/avatars/49a3f119b456ff94f28f09b2fe78bb18.svg","isPro":false,"fullname":"Heecheol Yun","user":"yoon6503","type":"user"},{"_id":"663073e4b108b8d71d4b6f32","avatarUrl":"/avatars/f66de2d81205397a29baf21169157bdd.svg","isPro":false,"fullname":"Inki Park","user":"seoharuss","type":"user"},{"_id":"6690a286181e2af45c742dd8","avatarUrl":"/avatars/511d0f86386e3b29a17b445d855b3aef.svg","isPro":false,"fullname":"Sohee Kim","user":"joyhee","type":"user"},{"_id":"66303ce3e1c93377db71efd5","avatarUrl":"/avatars/3d444dfb9799c9324c98cba893f4a10f.svg","isPro":false,"fullname":"Yoon Sik Park","user":"nooynoos","type":"user"},{"_id":"62845957b410bd779033759c","avatarUrl":"/avatars/4feef73c06f2f7de6abf7a4789ac13f9.svg","isPro":false,"fullname":"Doohyuk Jang","user":"jadohu","type":"user"},{"_id":"668ff6333bbfdee5f4f14a8a","avatarUrl":"/avatars/83037cfebaf75338296b23bff34c3b19.svg","isPro":true,"fullname":"Haechan Kim","user":"HaeChan0305","type":"user"},{"_id":"664f558b6f16bbd9a1481b59","avatarUrl":"/avatars/581c88278e1d8bba66ee3b764f6dd3ed.svg","isPro":false,"fullname":"Jo sungmin","user":"JoJosmin","type":"user"},{"_id":"65bbe7e2c084467aca4d0994","avatarUrl":"/avatars/a92ce1bcf144699adfda447423593967.svg","isPro":true,"fullname":"Jang","user":"Hyeongwon","type":"user"},{"_id":"62a4d58e81a4b10e93064ad6","avatarUrl":"/avatars/744d5cbc1745a26b816a458260aba050.svg","isPro":false,"fullname":"hangyulyoon","user":"hangyulmd","type":"user"},{"_id":"661cc2a9fbadbe6a9d18c8ff","avatarUrl":"/avatars/340e5eea14cff33e080a5f0e11e702dd.svg","isPro":false,"fullname":"Uigyu Kim","user":"Uigyu","type":"user"},{"_id":"6371ce78789970f7bc673234","avatarUrl":"/avatars/ba363e0b8fddee143244934be7bc6db0.svg","isPro":false,"fullname":"Donghyeon Cho","user":"hyeon9698","type":"user"},{"_id":"665ebae8bcbb98f60db0b4b1","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/665ebae8bcbb98f60db0b4b1/YTKM4qTZXh_2SeU8U7BfB.webp","isPro":false,"fullname":"Jiale Zhao","user":"Heisenburger2000","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"6475760c33192631bad2bb38","name":"kaist-ai","fullname":"KAIST AI","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/6469949654873f0043b09c22/aaZFiyXe1qR-Dmy_xq67m.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.16281.md","query":{}}">
Who Should Lead Decoding Now? Tracking Reliable Trajectories for Ensembling Masked Diffusion Language Models
Abstract
Masked diffusion language models exhibit unique decoding dynamics where reliable trajectories show stable confidence patterns, enabling iterative ensemble methods that transfer partially denoised sequences between models based on confidence evolution.
Masked Diffusion Language Models (MDLMs) have emerged as a distinct paradigm for sequence generation. As MDLMs become diverse in capabilities and knowledge coverage, an important question is how to combine their knowledge. Toward this, we first investigate the unique decoding dynamics of MDLMs. We find that successful generations exhibit stable confidence dynamics over answer-relevant positions, while unreliable trajectories can often be corrected by injecting promising intermediate states from other models. Guided by this observation, we propose TIE (Trajectory-based Iterative Ensembling), a knowledge fusion framework in which MDLMs iteratively identify reliable decoding trajectories and relay them across models. TIE tracks confidence dynamics over answer-relevant positions to determine which model currently follows a more reliable trajectory and selectively transfers partially denoised sequences across models. As the model on the more promising trajectory often changes across denoising steps, TIE allows different models to contribute complementary strengths at different stages of generation. Strong performance across diverse reasoning tasks, along with our analyses, suggests that TIE offers a practical approach to the underexplored problem of MDLM ensembling.
Community
We observe that successful MDLM generations exhibit stable confidence dynamics over answer-relevant positions, and that unreliable trajectories can often be corrected using promising intermediate states from other models. Building on this observation, we propose TIE, which enables knowledge fusion across heterogeneous MDLMs by iteratively identifying and relaying reliable decoding trajectories during generation.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2606.16281 in a model README.md to link it from this page.
Cite arxiv.org/abs/2606.16281 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2606.16281 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.