<a href=\"https://cdn-uploads.huggingface.co/production/uploads/6428762353b748123d4d65b4/N5yIB222AY3LA7TFq9GKV.png\" rel=\"nofollow\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/6428762353b748123d4d65b4/N5yIB222AY3LA7TFq9GKV.png\" alt=\"qual_halls_final\"></a></p>\n","updatedAt":"2026-06-04T02:16:06.469Z","author":{"_id":"6428762353b748123d4d65b4","avatarUrl":"/avatars/fda9e54ef5f660aa1ffb84a80840a3b7.svg","fullname":"Mahesh Bhosale","name":"mbhosale","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.4322895407676697},"editors":["mbhosale"],"editorAvatarUrls":["/avatars/fda9e54ef5f660aa1ffb84a80840a3b7.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.00377","authors":[{"_id":"6a20df5815100c5272a84663","name":"Mahesh Bhosale","hidden":false},{"_id":"6a20df5815100c5272a84664","name":"Naresh Kumar Devulapally","hidden":false},{"_id":"6a20df5815100c5272a84665","name":"Abdul Wasi","hidden":false},{"_id":"6a20df5815100c5272a84666","name":"Chau Pham","hidden":false},{"_id":"6a20df5815100c5272a84667","name":"Vishnu Suresh Lokhande","hidden":false},{"_id":"6a20df5815100c5272a84668","name":"David Doermann","hidden":false}],"publishedAt":"2026-05-29T00:00:00.000Z","submittedOnDailyAt":"2026-06-04T00:00:00.000Z","title":"Score-Control for Hallucination Reduction in Diffusion Models","submittedOnDailyBy":{"_id":"6428762353b748123d4d65b4","avatarUrl":"/avatars/fda9e54ef5f660aa1ffb84a80840a3b7.svg","isPro":false,"fullname":"Mahesh Bhosale","user":"mbhosale","type":"user","name":"mbhosale"},"summary":"Diffusion models have emerged as the backbone of modern generative AI, powering advances in vision, language, audio and other modalities. Despite their success, they suffer from hallucinations, implausible samples that lie outside the support of true data distribution, which degrade reliability and trust. In this work, we first empirically confirm previously proposed hypothesis that score smoothness causes hallucinations in Image Generation diffusion models and provide a density-based perspective. We further formalize this notion by linking the hallucinations probability mass to lipschitz constant of the learned score function. Motivated by this, we introduce a Variance-Guided Score Modulation (VSM) strategy that controls the score Jacobian, in turn reducing score smoothness and better approximating the ground truth score that decreases hallucinations. Empirical results on synthetic and real-world datasets demonstrate that our approach reduces hallucinations (up to ~25%) while maintaining high fidelity and diversity, providing a principled step toward more reliable diffusion-based image generation. We also propose two benchmark datasets with extreme semantic variation for systematic hallucination evaluation. Code and Datasets are publicly available at https://github.com/bhosalems/VSM.","upvotes":1,"discussionId":"6a20df5815100c5272a84669","githubRepo":"https://github.com/bhosalems/VSM","githubRepoAddedBy":"user","ai_summary":"Variance-Guided Score Modulation reduces hallucinations in diffusion models by controlling score function smoothness through Jacobian modulation while maintaining image quality.","ai_keywords":["diffusion models","hallucinations","score function","lipschitz constant","score Jacobian","Variance-Guided Score Modulation","image generation","semantic variation","hallucination evaluation"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","githubStars":0},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"699e9b5bb23a85efb33949fd","avatarUrl":"/avatars/80ba79a09d954d514cf1cb6af30512e7.svg","isPro":false,"fullname":"Song Qianyi","user":"songqianyi2025","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.00377.md"}">
Score-Control for Hallucination Reduction in Diffusion Models
Abstract
Variance-Guided Score Modulation reduces hallucinations in diffusion models by controlling score function smoothness through Jacobian modulation while maintaining image quality.
Diffusion models have emerged as the backbone of modern generative AI, powering advances in vision, language, audio and other modalities. Despite their success, they suffer from hallucinations, implausible samples that lie outside the support of true data distribution, which degrade reliability and trust. In this work, we first empirically confirm previously proposed hypothesis that score smoothness causes hallucinations in Image Generation diffusion models and provide a density-based perspective. We further formalize this notion by linking the hallucinations probability mass to lipschitz constant of the learned score function. Motivated by this, we introduce a Variance-Guided Score Modulation (VSM) strategy that controls the score Jacobian, in turn reducing score smoothness and better approximating the ground truth score that decreases hallucinations. Empirical results on synthetic and real-world datasets demonstrate that our approach reduces hallucinations (up to ~25%) while maintaining high fidelity and diversity, providing a principled step toward more reliable diffusion-based image generation. We also propose two benchmark datasets with extreme semantic variation for systematic hallucination evaluation. Code and Datasets are publicly available at https://github.com/bhosalems/VSM.
Community
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2606.00377 in a model README.md to link it from this page.
Cite arxiv.org/abs/2606.00377 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2606.00377 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.