We propose dRAE, a visual tokenizer built on Hyper-Spherical Quantization (HSQ). By assigning codes via cosine similarity and updating embeddings in tangent space, HSQ aligns the quantization objective with the intrinsic geometry of pre-trained vision encoder features (e.g., SigLIP2, DINOv2), achieving exceptionally high codebook utilization and strong semantic alignment.</p>\n","updatedAt":"2026-07-28T03:34:45.311Z","author":{"_id":"65ab7ed7d6b10af911dc8085","avatarUrl":"/avatars/0b033760f3a785ac82a1e913b28e9dec.svg","fullname":"Tianren Ma","name":"Vivre","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7475054860115051},"editors":["Vivre"],"editorAvatarUrls":["/avatars/0b033760f3a785ac82a1e913b28e9dec.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2607.22148","authors":[{"_id":"6a679d784534e38b50e9b1b1","name":"Tianren Ma","hidden":false},{"_id":"6a679d784534e38b50e9b1b2","name":"Lin Long","hidden":false},{"_id":"6a679d784534e38b50e9b1b3","name":"Chuyan Chen","hidden":false},{"_id":"6a679d784534e38b50e9b1b4","name":"Mu Zhang","hidden":false},{"_id":"6a679d784534e38b50e9b1b5","name":"Junbo Zhao","hidden":false},{"_id":"6a679d784534e38b50e9b1b6","name":"Tong Zhang","hidden":false},{"_id":"6a679d784534e38b50e9b1b7","name":"Qixiang Ye","hidden":false}],"publishedAt":"2026-07-24T09:47:32.000Z","submittedOnDailyAt":"2026-07-28T00:00:00.000Z","title":"dRAE: Representation Autoencoder with Hyper-Spherical Codes","submittedOnDailyBy":{"_id":"65ab7ed7d6b10af911dc8085","avatarUrl":"/avatars/0b033760f3a785ac82a1e913b28e9dec.svg","isPro":false,"fullname":"Tianren Ma","user":"Vivre","type":"user","name":"Vivre"},"summary":"In this work, we aim to discretize the high-dimensional visual representations to bridge the gap with language models - a non-trivial challenge, as existing quantization methods suffer from codebook collapse, failing to scale while preserving semantic coherence. We identify the root cause as metric mismatch: standard Euclidean codebook objectives are fundamentally misaligned with the anisotropic geometry of representation space, leading to codebook embeddings with high-variance magnitude scales and uneven angular distributions that hinder scalability. To address this, we propose Hyper-Spherical Quantization (HSQ), which decouples semantic content from feature magnitude via angular routing, preventing code assignment from being dominated by scale rather than meaning. The resulting discrete Representation Autoencoder (dRAE) achieves high-fidelity reconstruction while preserving semantic integrity and supporting scalable codebook budget. Extensive experiments demonstrate consistent performance gains as the vocabulary size scales to 131{,}072, along with 100\\% codebook utilization, simplified training pipeline, and strong performance across understanding and generation tasks.","upvotes":6,"discussionId":"6a679d784534e38b50e9b1b8","projectPage":"https://drae-hsq.github.io/","githubRepo":"https://github.com/martian422/dRAE","githubRepoAddedBy":"user","githubStars":3,"organization":{"_id":"67c1d682826160b28f778510","name":"antgroup","fullname":"Ant Group","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/662e1f9da266499277937d33/7VcPHdLSGlged3ixK1dys.jpeg"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"65ab7ed7d6b10af911dc8085","avatarUrl":"/avatars/0b033760f3a785ac82a1e913b28e9dec.svg","isPro":false,"fullname":"Tianren Ma","user":"Vivre","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"6a326ac31adddef183b57e5f","avatarUrl":"/avatars/ba334cd23ddc80f26f45bb6d10de0fcf.svg","isPro":false,"fullname":"Chuzhong","user":"VanceChu","type":"user"},{"_id":"652b9b512aa5b27c77e303bf","avatarUrl":"/avatars/5f61acb4b5327e2d188b5b399a457bc0.svg","isPro":false,"fullname":"WangYB","user":"WEBing","type":"user"},{"_id":"6641b963b5f815a3eaba476e","avatarUrl":"/avatars/65bdfeb3fbc0866661c76e1213609e96.svg","isPro":false,"fullname":"Tianyu Wang","user":"wangtyUCAS","type":"user"},{"_id":"65aa76b1cb5b4fb08ecb087c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65aa76b1cb5b4fb08ecb087c/0_x6Zb-H36XfugcsQ80Zh.jpeg","isPro":false,"fullname":"Liu Yue","user":"Mzero17","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"67c1d682826160b28f778510","name":"antgroup","fullname":"Ant Group","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/662e1f9da266499277937d33/7VcPHdLSGlged3ixK1dys.jpeg"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2607/2607.22148.md","query":{}}">

Get this paper in your agent:

Don't have the latest CLI?

Cite arxiv.org/abs/2607.22148 in a model README.md to link it from this page.

Cite arxiv.org/abs/2607.22148 in a dataset README.md to link it from this page.

Cite arxiv.org/abs/2607.22148 in a Space README.md to link it from this page.