<strong>Most quality degradation occurs before latent generation even begins.</strong></p>\n<p>We present a staged diagnostic protocol for compressed text generation that separates codec reconstruction loss, latent-generation loss, and improvements visible only in latent-space proxies.</p>\n<p>In a controlled TinyStories 64→16 setup:</p>\n<ul>\n<li>codec reconstruction alone increases median external PPL from <strong>15.17 to 27.36</strong> (+80.4%) and p95 from <strong>25.10 to 98.91</strong> (+294.1%);</li>\n<li>code-space MDLM outperforms the token-space baseline, reducing mean, median, and p95 PPL by <strong>32.9%, 30.9%, and 36.6%</strong>, respectively;</li>\n<li>geometry-aware regularization improves local latent diagnostics but does not improve decoded-text quality.</li>\n</ul>\n<p><strong>Practical takeaway:</strong> audit codec reconstruction before spending compute on the latent generator, and validate every latent-space improvement on the final decoded text.</p>\n<p>This is a methodological case study rather than a universal ranking of codecs or diffusion models.</p>\n","updatedAt":"2026-07-29T09:38:56.532Z","author":{"_id":"63a041c56b087d7413be69e8","avatarUrl":"/avatars/66593b5a09540f343377e8dffe9e0100.svg","fullname":"Alexey Gavrilov","name":"aogavrilov","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7814592123031616},"editors":["aogavrilov"],"editorAvatarUrls":["/avatars/66593b5a09540f343377e8dffe9e0100.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2607.24176","authors":[{"_id":"6a68414773f69d5af2bec901","user":{"_id":"63a041c56b087d7413be69e8","avatarUrl":"/avatars/66593b5a09540f343377e8dffe9e0100.svg","isPro":false,"fullname":"Alexey Gavrilov","user":"aogavrilov","type":"user","name":"aogavrilov"},"name":"Alexey Gavrilov","status":"claimed_verified","statusLastChangedAt":"2026-07-29T08:45:04.349Z","hidden":false},{"_id":"6a68414773f69d5af2bec902","name":"Alan-Barsag Gazzaev","hidden":false},{"_id":"6a68414773f69d5af2bec903","name":"Sergey Muravyov","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/63a041c56b087d7413be69e8/DOAdqFThlRyhC4hZD2tub.png","https://cdn-uploads.huggingface.co/production/uploads/63a041c56b087d7413be69e8/kemZe_bcmnf9BxNL8GKCq.png"],"publishedAt":"2026-07-27T00:00:00.000Z","submittedOnDailyAt":"2026-07-29T00:00:00.000Z","title":"Where Quality Breaks in Compressed Short-Text Generation: Staged Bottleneck Localization","submittedOnDailyBy":{"_id":"63a041c56b087d7413be69e8","avatarUrl":"/avatars/66593b5a09540f343377e8dffe9e0100.svg","isPro":false,"fullname":"Alexey Gavrilov","user":"aogavrilov","type":"user","name":"aogavrilov"},"summary":"Compressed short-text generators can fail in two different places: the codec may discard information before generation starts, or the latent generator may produce weak codes. Without separating these failure modes, researchers can spend compute improving the wrong component. We study this problem in a controlled 64-to-16 TinyStories case study built from a hierarchical VQ-VAE-2 codec and a masked discrete diffusion generator (MDLM). We use a staged validation protocol that separates codec reconstruction fidelity, latent generation quality, and auxiliary latent diagnostics under one shared external GPT-2 scorer, while reporting complementary semantic metrics for the geometry study. In the tested configuration, codec reconstruction alone raises median external perplexity from 15.17 to 27.36 (+80.4%) and p95 from 25.10 to 98.91 (+294.1%), showing that the dominant quality loss appears before latent generation begins. Under the same scorer, code-space MDLM remains materially stronger than token-space diffusion, reducing mean, median, and p95 by 32.9%, 30.9%, and 36.6%, respectively. Geometry-aware regularization improves local latent proxies but does not improve decoded-text metrics in the available runs. The contribution is methodological rather than algorithmic: the paper presents a reusable staged diagnosis for one concrete pipeline and shows that, in this setting, codec fidelity rather than latent denoising sets the practical quality ceiling.","upvotes":3,"discussionId":"6a68414773f69d5af2bec904","projectPage":"https://aogavrilov.com/publications/where-quality-breaks/","organization":{"_id":"61b8da8ed48e2b6a4d2f5530","name":"ITMO","fullname":"ITMO University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/fBvTjWt5kZM802VUpqCno.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"63a041c56b087d7413be69e8","avatarUrl":"/avatars/66593b5a09540f343377e8dffe9e0100.svg","isPro":false,"fullname":"Alexey Gavrilov","user":"aogavrilov","type":"user"},{"_id":"63654b82ad2c9e8b672d8a4b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1667661623748-63654b82ad2c9e8b672d8a4b.jpeg","isPro":false,"fullname":"Alan Gazzaev","user":"AlanRobotics","type":"user"},{"_id":"63ac5701c21e60a3e9b58aa7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63ac5701c21e60a3e9b58aa7/g6EX7diOpuA94R2ab-rZC.png","isPro":true,"fullname":"Dipankar Sarkar","user":"dipankarsarkar","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"61b8da8ed48e2b6a4d2f5530","name":"ITMO","fullname":"ITMO University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/fBvTjWt5kZM802VUpqCno.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2607/2607.24176.md","query":{}}">

Get this paper in your agent:

Don't have the latest CLI?

Cite arxiv.org/abs/2607.24176 in a model README.md to link it from this page.

Cite arxiv.org/abs/2607.24176 in a dataset README.md to link it from this page.

Cite arxiv.org/abs/2607.24176 in a Space README.md to link it from this page.