We show that standard implementations of Uniform Diffusion Models (Multinomial Diffusion, USDM, Duo, GIDD...) do not learn a denoiser but instead a \"leave-one-out\" denoiser. We use this characterization to improve training and design new inference-time tools such as predictor correctors that substantially improve the generative frontier for diffusion language models. In addition, we show that uniform and masked diffusion are two sides of the same coin and propose a new parameterization of uniform diffusion models, where the denoiser is closer to masked diffusion.</p>\n","updatedAt":"2026-05-29T12:30:54.441Z","author":{"_id":"699b2e1423171de52d2e2275","avatarUrl":"/avatars/c7eb7dfd880b7f4afd238b666e5d9c57.svg","fullname":"Samson Gourevitch","name":"samsongourevitch","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8357846140861511},"editors":["samsongourevitch"],"editorAvatarUrls":["/avatars/c7eb7dfd880b7f4afd238b666e5d9c57.svg"],"reactions":[],"isReport":false}},{"id":"6a1a4105a311d2bf1543ccb3","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":359,"isUserFollowing":false},"createdAt":"2026-05-30T01:44:37.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [Sampling from Flow Language Models via Marginal-Conditioned Bridges](https://huggingface.co/papers/2605.13681) (2026)\n* [Support Before Frequency in Discrete Diffusion](https://huggingface.co/papers/2605.13999) (2026)\n* [Interpolating Discrete Diffusion Models with Controllable Resampling](https://huggingface.co/papers/2604.17310) (2026)\n* [Backdooring Masked Diffusion Language Models](https://huggingface.co/papers/2605.19262) (2026)\n* [Simple Self-Conditioning Adaptation for Masked Diffusion Models](https://huggingface.co/papers/2604.26985) (2026)\n* [How to Train Your Latent Diffusion Language Model Jointly With the Latent Space](https://huggingface.co/papers/2605.07933) (2026)\n* [Understanding and Accelerating the Training of Masked Diffusion Language Models](https://huggingface.co/papers/2605.13026) (2026)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"<p>This is an automated message from the <a href=\"https://huggingface.co/librarian-bots\">Librarian Bot</a>. I found the following papers similar to this paper. </p>\n<p>The following papers were recommended by the Semantic Scholar API </p>\n<ul>\n<li><a href=\"https://huggingface.co/papers/2605.13681\">Sampling from Flow Language Models via Marginal-Conditioned Bridges</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.13999\">Support Before Frequency in Discrete Diffusion</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2604.17310\">Interpolating Discrete Diffusion Models with Controllable Resampling</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.19262\">Backdooring Masked Diffusion Language Models</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2604.26985\">Simple Self-Conditioning Adaptation for Masked Diffusion Models</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.07933\">How to Train Your Latent Diffusion Language Model Jointly With the Latent Space</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.13026\">Understanding and Accelerating the Training of Masked Diffusion Language Models</a> (2026)</li>\n</ul>\n<p> Please give a thumbs up to this comment if you found it helpful!</p>\n<p> If you want recommendations for any Paper on Hugging Face checkout <a href=\"https://huggingface.co/spaces/librarian-bots/recommend_similar_papers\">this</a> Space</p>\n<p> You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: <code><span class=\"SVELTE_PARTIAL_HYDRATER contents\" data-target=\"UserMention\" data-props=\"{"user":"librarian-bot"}\"><span class=\"inline-block\"><span class=\"contents\"><a href=\"/librarian-bot\">@<span class=\"underline\">librarian-bot</span></a></span> </span></span> recommend</code></p>\n","updatedAt":"2026-05-30T01:44:37.420Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":359,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7176114320755005},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2605.22765","authors":[{"_id":"6a101d9ea53a61ce2e422f1b","name":"Samson Gourevitch","hidden":false},{"_id":"6a101d9ea53a61ce2e422f1c","name":"Yazid Janati","hidden":false},{"_id":"6a101d9ea53a61ce2e422f1d","name":"Dario Shariatian","hidden":false},{"_id":"6a101d9ea53a61ce2e422f1e","name":"Umut Simsekli","hidden":false},{"_id":"6a101d9ea53a61ce2e422f1f","name":"Eric Moulines","hidden":false},{"_id":"6a101d9ea53a61ce2e422f20","name":"Eric P. Xing","hidden":false},{"_id":"6a101d9ea53a61ce2e422f21","name":"Alain Durmus","hidden":false}],"publishedAt":"2026-05-21T00:00:00.000Z","submittedOnDailyAt":"2026-05-29T00:00:00.000Z","title":"Uniform Diffusion Models Revisited: Leave-One-Out Denoiser and Absorbing State Reformulation","submittedOnDailyBy":{"_id":"699b2e1423171de52d2e2275","avatarUrl":"/avatars/c7eb7dfd880b7f4afd238b666e5d9c57.svg","isPro":false,"fullname":"Samson Gourevitch","user":"samsongourevitch","type":"user","name":"samsongourevitch"},"summary":"Discrete diffusion models are often trained through clean-data prediction, but the prediction can be used in different ways to define the reverse dynamics. In Masked Diffusion Models (MDM) these choices largely coincide, whereas in Uniform Diffusion Models (UDM) they do not. We show that the standard plug-in bridge parameterization for UDM is not optimized by the denoising posterior, but by a leave-one-out posterior that predicts each clean token without using its own noisy observation. This identifies a mismatch between the plug-in ELBO and the usual cross-entropy denoising objective. We characterize the leave-one-out target and derive exact conversions between the denoiser, the leave-one-out posterior, and the score. These conversions allow us to disentangle parameterization and training objective. Our results also lead to inference improvements without any additional training through an informed predictor-corrector sampler and improved temperature sampling based on the leave-one-out predictor.\n We further introduce an absorbing-state reformulation of uniform diffusion that preserves the UDM joint law while decomposing it into masked-diffusion-like sampling operations, with simpler denoising posteriors, carry-over unmasking, and a natural remasking mechanism. On language modeling, leave-one-out parameterizations consistently improve UDM generation, while the absorbing construction matches or surpasses masked diffusion. These results suggest that the empirical gap between masked and uniform diffusion is driven less by the choice of marginals themselves than by parameterization and sampling design. The code and models can be found at https://github.com/samsongourevitch/rev_udm.","upvotes":1,"discussionId":"6a101d9ea53a61ce2e422f22","githubRepo":"https://github.com/samsongourevitch/rev_udm","githubRepoAddedBy":"user","githubStars":4},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"699b2e1423171de52d2e2275","avatarUrl":"/avatars/c7eb7dfd880b7f4afd238b666e5d9c57.svg","isPro":false,"fullname":"Samson Gourevitch","user":"samsongourevitch","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0}">
Uniform Diffusion Models Revisited: Leave-One-Out Denoiser and Absorbing State Reformulation
Abstract
Discrete diffusion models are often trained through clean-data prediction, but the prediction can be used in different ways to define the reverse dynamics. In Masked Diffusion Models (MDM) these choices largely coincide, whereas in Uniform Diffusion Models (UDM) they do not. We show that the standard plug-in bridge parameterization for UDM is not optimized by the denoising posterior, but by a leave-one-out posterior that predicts each clean token without using its own noisy observation. This identifies a mismatch between the plug-in ELBO and the usual cross-entropy denoising objective. We characterize the leave-one-out target and derive exact conversions between the denoiser, the leave-one-out posterior, and the score. These conversions allow us to disentangle parameterization and training objective. Our results also lead to inference improvements without any additional training through an informed predictor-corrector sampler and improved temperature sampling based on the leave-one-out predictor.
We further introduce an absorbing-state reformulation of uniform diffusion that preserves the UDM joint law while decomposing it into masked-diffusion-like sampling operations, with simpler denoising posteriors, carry-over unmasking, and a natural remasking mechanism. On language modeling, leave-one-out parameterizations consistently improve UDM generation, while the absorbing construction matches or surpasses masked diffusion. These results suggest that the empirical gap between masked and uniform diffusion is driven less by the choice of marginals themselves than by parameterization and sampling design. The code and models can be found at https://github.com/samsongourevitch/rev_udm.
Community
We show that standard implementations of Uniform Diffusion Models (Multinomial Diffusion, USDM, Duo, GIDD...) do not learn a denoiser but instead a "leave-one-out" denoiser. We use this characterization to improve training and design new inference-time tools such as predictor correctors that substantially improve the generative frontier for diffusion language models. In addition, we show that uniform and masked diffusion are two sides of the same coin and propose a new parameterization of uniform diffusion models, where the denoiser is closer to masked diffusion.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2605.22765 in a model README.md to link it from this page.
Cite arxiv.org/abs/2605.22765 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2605.22765 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.