General solution for accelerating visual geometry transformers.</p>\n","updatedAt":"2026-05-25T05:04:56.014Z","author":{"_id":"64361417a4bd75c62cc1e534","avatarUrl":"/avatars/edf6ac3fb9bd63ceb78f3b95254e6b19.svg","fullname":"Shuhong Zheng","name":"ShuhongZheng","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7144063115119934},"editors":["ShuhongZheng"],"editorAvatarUrls":["/avatars/edf6ac3fb9bd63ceb78f3b95254e6b19.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2605.23892","authors":[{"_id":"6a13d7584d9e8d8602d20327","name":"Shuhong Zheng","hidden":false},{"_id":"6a13d7584d9e8d8602d20328","name":"Michael Oechsle","hidden":false},{"_id":"6a13d7584d9e8d8602d20329","name":"Erik Sandström","hidden":false},{"_id":"6a13d7584d9e8d8602d2032a","name":"Marie-Julie Rakotosaona","hidden":false},{"_id":"6a13d7584d9e8d8602d2032b","name":"Federico Tombari","hidden":false},{"_id":"6a13d7584d9e8d8602d2032c","name":"Igor Gilitschenski","hidden":false}],"publishedAt":"2026-05-22T00:00:00.000Z","submittedOnDailyAt":"2026-05-25T00:00:00.000Z","title":"Good Token Hunting: A Hitchhiker's Guide to Token Selection for Visual Geometry Transformers","submittedOnDailyBy":{"_id":"64361417a4bd75c62cc1e534","avatarUrl":"/avatars/edf6ac3fb9bd63ceb78f3b95254e6b19.svg","isPro":false,"fullname":"Shuhong Zheng","user":"ShuhongZheng","type":"user","name":"ShuhongZheng"},"summary":"Visual geometry transformers have become powerful architectures for multi-view 3D reconstruction, enabling joint prediction of multiple 3D attributes in a feed-forward manner. However, their computational cost grows quadratically with the input sequence length due to the global attention layers inside these models. This limits both their scalability and efficiency. In this work, we address this challenge with a simple yet general strategy: restricting the number of key/value tokens that each query interacts with during global attention. To achieve effective token selection, we introduce a two-stage framework. First, an inter-frame selection step operates at the frame level to identify frames that should be preserved. Second, an intra-frame selection step further discards more redundant tokens within the selected frames. Our analysis highlights the advantage of a diversity-based strategy for inter-frame selection, which ensures broad coverage of the scene. For intra-frame selection, we show that layer-aware sparsification is necessary, with the selection process guided by the entropy of the global attention pattern. Our approach offers a superior speed-accuracy trade-off compared to existing solutions. Extensive experiments show that it accelerates visual geometry transformers by over 85% for scenes with 500 images while maintaining, or even improving, baseline performance, which hints that how our token selection strategy can play a crucial role in future applications of visual geometry transformers. Our project website is available at https://zsh2000.github.io/good-token-hunting.github.io.","upvotes":2,"discussionId":"6a13d7594d9e8d8602d2032d","projectPage":"https://zsh2000.github.io/good-token-hunting.github.io/","githubRepo":"https://github.com/zsh2000/gotohunt","githubRepoAddedBy":"user","ai_summary":"Visual geometry transformers are accelerated through a two-stage token selection framework that reduces computational costs while maintaining performance.","ai_keywords":["visual geometry transformers","multi-view 3D reconstruction","global attention layers","token selection","inter-frame selection","intra-frame selection","layer-aware sparsification","entropy of global attention pattern"],"githubStars":4,"organization":{"_id":"62c5000b4d3cf26ce7c62822","name":"uoft","fullname":"University of Toronto","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1657077766523-62c4ff85cb7033fd49b7a559.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"64361417a4bd75c62cc1e534","avatarUrl":"/avatars/edf6ac3fb9bd63ceb78f3b95254e6b19.svg","isPro":false,"fullname":"Shuhong Zheng","user":"ShuhongZheng","type":"user"},{"_id":"68e41658ed521a20f060f103","avatarUrl":"/avatars/e35cb262601a5ee932fcd3cbbe10be5e.svg","isPro":false,"fullname":"Shuhong Zheng","user":"zsh2000","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"62c5000b4d3cf26ce7c62822","name":"uoft","fullname":"University of Toronto","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1657077766523-62c4ff85cb7033fd49b7a559.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2605/2605.23892.md"}">
Good Token Hunting: A Hitchhiker's Guide to Token Selection for Visual Geometry Transformers
Abstract
Visual geometry transformers are accelerated through a two-stage token selection framework that reduces computational costs while maintaining performance.
AI-generated summary
Visual geometry transformers have become powerful architectures for multi-view 3D reconstruction, enabling joint prediction of multiple 3D attributes in a feed-forward manner. However, their computational cost grows quadratically with the input sequence length due to the global attention layers inside these models. This limits both their scalability and efficiency. In this work, we address this challenge with a simple yet general strategy: restricting the number of key/value tokens that each query interacts with during global attention. To achieve effective token selection, we introduce a two-stage framework. First, an inter-frame selection step operates at the frame level to identify frames that should be preserved. Second, an intra-frame selection step further discards more redundant tokens within the selected frames. Our analysis highlights the advantage of a diversity-based strategy for inter-frame selection, which ensures broad coverage of the scene. For intra-frame selection, we show that layer-aware sparsification is necessary, with the selection process guided by the entropy of the global attention pattern. Our approach offers a superior speed-accuracy trade-off compared to existing solutions. Extensive experiments show that it accelerates visual geometry transformers by over 85% for scenes with 500 images while maintaining, or even improving, baseline performance, which hints that how our token selection strategy can play a crucial role in future applications of visual geometry transformers. Our project website is available at https://zsh2000.github.io/good-token-hunting.github.io.
Community
General solution for accelerating visual geometry transformers.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2605.23892 in a model README.md to link it from this page.
Cite arxiv.org/abs/2605.23892 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2605.23892 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.