github:<a href=\"https://github.com/Zyvpeng/VisCo\" rel=\"nofollow\">https://github.com/Zyvpeng/VisCo</a></p>\n","updatedAt":"2026-07-26T08:23:23.408Z","author":{"_id":"672b68fcf437cca3d6259dfd","avatarUrl":"/avatars/939903ae7b5adeaa297e5e0d83596e8a.svg","fullname":"ZhengYupeng","name":"zpatrick","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.6295773386955261},"editors":["zpatrick"],"editorAvatarUrls":["/avatars/939903ae7b5adeaa297e5e0d83596e8a.svg"],"reactions":[],"isReport":false}},{"id":"6a66bf5d382b3e981feedb1a","author":{"_id":"672b68fcf437cca3d6259dfd","avatarUrl":"/avatars/939903ae7b5adeaa297e5e0d83596e8a.svg","fullname":"ZhengYupeng","name":"zpatrick","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false},"createdAt":"2026-07-27T02:15:57.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Accepted by ACM MM 2026","html":"<p>Accepted by ACM MM 2026</p>\n","updatedAt":"2026-07-27T02:15:57.223Z","author":{"_id":"672b68fcf437cca3d6259dfd","avatarUrl":"/avatars/939903ae7b5adeaa297e5e0d83596e8a.svg","fullname":"ZhengYupeng","name":"zpatrick","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9387334585189819},"editors":["zpatrick"],"editorAvatarUrls":["/avatars/939903ae7b5adeaa297e5e0d83596e8a.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2607.12756","authors":[{"_id":"6a65c3baab9cdf9be579485a","user":{"_id":"672b68fcf437cca3d6259dfd","avatarUrl":"/avatars/939903ae7b5adeaa297e5e0d83596e8a.svg","isPro":false,"fullname":"ZhengYupeng","user":"zpatrick","type":"user","name":"zpatrick"},"name":"Yupeng Zheng","status":"claimed_verified","statusLastChangedAt":"2026-07-26T08:45:04.134Z","hidden":false},{"_id":"6a65c3baab9cdf9be579485b","name":"Kai Zou","hidden":false},{"_id":"6a65c3baab9cdf9be579485c","name":"Bin Liu","hidden":false},{"_id":"6a65c3baab9cdf9be579485d","name":"Nenghai Yu","hidden":false}],"publishedAt":"2026-07-14T00:00:00.000Z","submittedOnDailyAt":"2026-07-27T00:00:00.000Z","title":"VisCo: Leveraging Large Language Models as Intrinsic Encoders for Visual Token Compression","submittedOnDailyBy":{"_id":"672b68fcf437cca3d6259dfd","avatarUrl":"/avatars/939903ae7b5adeaa297e5e0d83596e8a.svg","isPro":false,"fullname":"ZhengYupeng","user":"zpatrick","type":"user","name":"zpatrick"},"summary":"Vision-language models (VLMs) process large numbers of visual tokens, resulting in substantial inference latency and memory overhead. This has motivated extensive research on visual token compression. While training-free strategies rely on heuristic metrics and suffer significant performance degradation under high compression ratios, many training-based methods introduce external compression modules that force the VLM backbone to adapt, incurring substantial retraining cost and compromising VLMs' priors. Effective visual token compression hinges on strong information encoding, a capability already present in pretrained VLMs but underutilized by existing approaches. Motivated by this, we propose VisCo, a training-efficient self-compression framework that reuses the pretrained VLM itself as an intrinsic compressor. VisCo is a parameter-sharing autoencoder that compresses visual information using a small set of memory tokens and transfers hierarchical information from encoding to decoding. Experiments show that VisCo surpasses prior methods across all evaluated compression ratios, with larger gains under more aggressive compression, and remains stable even in the extreme single-token setting. Moreover, when combined with the original visual tokens, the learned memory tokens can even improve the base model, suggesting that VisCo captures complementary representations beyond compression.","upvotes":1,"discussionId":"6a65c3baab9cdf9be579485e","githubRepo":"https://github.com/Zyvpeng/VisCo","githubRepoAddedBy":"user","githubStars":11},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"672b68fcf437cca3d6259dfd","avatarUrl":"/avatars/939903ae7b5adeaa297e5e0d83596e8a.svg","isPro":false,"fullname":"ZhengYupeng","user":"zpatrick","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"query":{}}">
VisCo: Leveraging Large Language Models as Intrinsic Encoders for Visual Token Compression
Abstract
Vision-language models (VLMs) process large numbers of visual tokens, resulting in substantial inference latency and memory overhead. This has motivated extensive research on visual token compression. While training-free strategies rely on heuristic metrics and suffer significant performance degradation under high compression ratios, many training-based methods introduce external compression modules that force the VLM backbone to adapt, incurring substantial retraining cost and compromising VLMs' priors. Effective visual token compression hinges on strong information encoding, a capability already present in pretrained VLMs but underutilized by existing approaches. Motivated by this, we propose VisCo, a training-efficient self-compression framework that reuses the pretrained VLM itself as an intrinsic compressor. VisCo is a parameter-sharing autoencoder that compresses visual information using a small set of memory tokens and transfers hierarchical information from encoding to decoding. Experiments show that VisCo surpasses prior methods across all evaluated compression ratios, with larger gains under more aggressive compression, and remains stable even in the extreme single-token setting. Moreover, when combined with the original visual tokens, the learned memory tokens can even improve the base model, suggesting that VisCo captures complementary representations beyond compression.
Community
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2607.12756 in a model README.md to link it from this page.
Cite arxiv.org/abs/2607.12756 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2607.12756 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.