IDEAL: In-DEpth ALignment Makes A Discrete Representation AutoEncoder</p>\n","updatedAt":"2026-06-12T06:33:49.516Z","author":{"_id":"64651db3611ae99d14d392ea","avatarUrl":"/avatars/b818dc0dddc999758ab5737d5053e8c3.svg","fullname":"YitongChen (SII)","name":"Row11n","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.48939964175224304},"editors":["Row11n"],"editorAvatarUrls":["/avatars/b818dc0dddc999758ab5737d5053e8c3.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.11096","authors":[{"_id":"6a2ba7e04957fcdd3aac07be","user":{"_id":"64651db3611ae99d14d392ea","avatarUrl":"/avatars/b818dc0dddc999758ab5737d5053e8c3.svg","isPro":false,"fullname":"YitongChen (SII)","user":"Row11n","type":"user","name":"Row11n"},"name":"Yitong Chen","status":"claimed_verified","statusLastChangedAt":"2026-06-12T06:56:12.275Z","hidden":false},{"_id":"6a2ba7e04957fcdd3aac07bf","name":"Zijie Diao","hidden":false},{"_id":"6a2ba7e04957fcdd3aac07c0","name":"Junke Wang","hidden":false},{"_id":"6a2ba7e04957fcdd3aac07c1","name":"Lingyu Kong","hidden":false},{"_id":"6a2ba7e04957fcdd3aac07c2","name":"Yixuan Ren","hidden":false},{"_id":"6a2ba7e04957fcdd3aac07c3","name":"Bo He","hidden":false},{"_id":"6a2ba7e04957fcdd3aac07c4","name":"Yu-Gang Jiang","hidden":false},{"_id":"6a2ba7e04957fcdd3aac07c5","name":"Zuxuan Wu","hidden":false}],"publishedAt":"2026-06-09T00:00:00.000Z","submittedOnDailyAt":"2026-06-12T00:00:00.000Z","title":"IDEAL: In-DEpth ALignment Makes A Discrete Representation AutoEncoder","submittedOnDailyBy":{"_id":"64651db3611ae99d14d392ea","avatarUrl":"/avatars/b818dc0dddc999758ab5737d5053e8c3.svg","isPro":false,"fullname":"YitongChen (SII)","user":"Row11n","type":"user","name":"Row11n"},"summary":"Built on pretrained vision foundation models (VFMs), representation autoencoders (RAEs) have recently emerged as a promising approach for constructing semantically rich latent spaces for image generation. However, their reconstruction quality often remains suboptimal, largely because deep VFM representations do not preserve sufficient fine-grained visual detail. This limitation becomes even more severe after discretization, where missing low-level information is difficult to recover. In fact, we observe that shallow VFM features retain considerably richer local appearance and structural detail, which complements the high-level semantics carried by deep features used in existing RAEs. Motivated by this complementary property, we propose Ideal, an In-depth Alignment framework for discrete representation autoencoding. By jointly aligning quantized tokens with both shallow and deep VFM features, Ideal enables the resulting discrete visual tokens to preserve both visual fidelity and rich semantics. Extensive experiments demonstrate that Ideal yields superior reconstruction performance, achieving 0.61 rFID on ImageNet and outperforming the previous best method by 0.28. When used for autoregressive image generation, Ideal further produces a gFID of 1.89, establishing a new state of the art for autoregressive image generation.","upvotes":1,"discussionId":"6a2ba7e04957fcdd3aac07c6","githubRepo":"https://github.com/Row11n/IDEAL","githubRepoAddedBy":"user","ai_summary":"Representation autoencoders using deep learning frameworks can improve image reconstruction quality by combining shallow and deep visual feature representations for better semantic richness and visual fidelity.","ai_keywords":["representation autoencoders","vision foundation models","discrete representation autoencoding","quantized tokens","visual fidelity","autoregressive image generation","rFID","gFID"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","githubStars":5},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"64651db3611ae99d14d392ea","avatarUrl":"/avatars/b818dc0dddc999758ab5737d5053e8c3.svg","isPro":false,"fullname":"YitongChen (SII)","user":"Row11n","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.11096.md","query":{}}">
IDEAL: In-DEpth ALignment Makes A Discrete Representation AutoEncoder
Abstract
Representation autoencoders using deep learning frameworks can improve image reconstruction quality by combining shallow and deep visual feature representations for better semantic richness and visual fidelity.
Built on pretrained vision foundation models (VFMs), representation autoencoders (RAEs) have recently emerged as a promising approach for constructing semantically rich latent spaces for image generation. However, their reconstruction quality often remains suboptimal, largely because deep VFM representations do not preserve sufficient fine-grained visual detail. This limitation becomes even more severe after discretization, where missing low-level information is difficult to recover. In fact, we observe that shallow VFM features retain considerably richer local appearance and structural detail, which complements the high-level semantics carried by deep features used in existing RAEs. Motivated by this complementary property, we propose Ideal, an In-depth Alignment framework for discrete representation autoencoding. By jointly aligning quantized tokens with both shallow and deep VFM features, Ideal enables the resulting discrete visual tokens to preserve both visual fidelity and rich semantics. Extensive experiments demonstrate that Ideal yields superior reconstruction performance, achieving 0.61 rFID on ImageNet and outperforming the previous best method by 0.28. When used for autoregressive image generation, Ideal further produces a gFID of 1.89, establishing a new state of the art for autoregressive image generation.
Community
IDEAL: In-DEpth ALignment Makes A Discrete Representation AutoEncoder
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2606.11096 in a model README.md to link it from this page.
Cite arxiv.org/abs/2606.11096 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2606.11096 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.