Stream3D-VLM is an online 3D vision-language model that supports real-time spatial understanding and interaction directly from streaming video. By incrementally integrating geometry priors and employing geometry-adaptive voxel compression, our approach enables efficient and continuous 3D scene comprehension without requiring offline processing or complete scene observations.</p>\n","updatedAt":"2026-06-08T04:13:29.082Z","author":{"_id":"652d3023cdb2a91205709b6a","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/652d3023cdb2a91205709b6a/x-MaRkBwSdZ0xIYDJg09u.jpeg","fullname":"Hanxun Yu","name":"JonnyYu828","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false}},"numEdits":1,"identifiedLanguage":{"language":"en","probability":0.7467535734176636},"editors":["JonnyYu828"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/652d3023cdb2a91205709b6a/x-MaRkBwSdZ0xIYDJg09u.jpeg"],"reactions":[{"reaction":"🔥","users":["JonnyYu828"],"count":1},{"reaction":"🚀","users":["JonnyYu828"],"count":1},{"reaction":"👍","users":["JonnyYu828"],"count":1}],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.06891","authors":[{"_id":"6a263b8ee4c258a02949203d","name":"Hanxun Yu","hidden":false},{"_id":"6a263b8ee4c258a02949203e","name":"Xuan Qu","hidden":false},{"_id":"6a263b8ee4c258a02949203f","name":"Lei Ke","hidden":false},{"_id":"6a263b8ee4c258a029492040","name":"Boqiang Zhang","hidden":false},{"_id":"6a263b8ee4c258a029492041","name":"Yuxin Wang","hidden":false},{"_id":"6a263b8ee4c258a029492042","name":"Jianke Zhu","hidden":false},{"_id":"6a263b8ee4c258a029492043","name":"Dong Yu","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/652d3023cdb2a91205709b6a/920b49ousFydb1FxOwMED.mp4"],"publishedAt":"2026-06-05T00:00:00.000Z","submittedOnDailyAt":"2026-06-08T00:00:00.000Z","title":"Stream3D-VLM: Online 3D Spatial Understanding with Incremental Geometry Priors","submittedOnDailyBy":{"_id":"652d3023cdb2a91205709b6a","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/652d3023cdb2a91205709b6a/x-MaRkBwSdZ0xIYDJg09u.jpeg","isPro":false,"fullname":"Hanxun Yu","user":"JonnyYu828","type":"user","name":"JonnyYu828"},"summary":"Despite advances in 3D scene understanding, existing 3D Large Multimodal Models operate in offline settings, requiring complete scene observations or predefined video clips. In this paper, we present an online 3D vision-language model that enables real-time spatial understanding from streaming video. Our approach adopts an autoregressive streaming control modeling based on the LLM's next-token prediction objective to learn when to respond, and employs a lightweight Visual-Spatial Feature Integration (VSFI) module to incrementally inject temporally aligned geometry priors into the visual stream. To alleviate long-context decoding overhead, we propose a plug-and-play Geometry-Adaptive Voxel Compression (GAVC) module for efficient visual token compression. To address the scarcity of streaming 3D-language data, we further develop a scalable data generation pipeline that curates over 1M online spatio-temporal 3D QA pairs and establishes a comprehensive benchmark spanning 29 tasks. Extensive experiments show that our approach significantly outperforms both proprietary and open-source models across online and offline 3D spatial understanding, reasoning, and grounding tasks. The project page is available at https://stream3d-vlm.github.io/","upvotes":2,"discussionId":"6a263b8ee4c258a029492044","projectPage":"https://stream3d-vlm.github.io/","githubRepo":"https://github.com/hanxunyu/Stream3D-VLM","githubRepoAddedBy":"user","ai_summary":"An online 3D vision-language model enables real-time spatial understanding from streaming video using autoregressive control modeling and efficient visual token compression.","ai_keywords":["3D Large Multimodal Models","autoregressive streaming control modeling","next-token prediction objective","Visual-Spatial Feature Integration","Geometry-Adaptive Voxel Compression","spatio-temporal 3D QA pairs"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","githubStars":3,"organization":{"_id":"6645f953c39288df638dbdd5","name":"Tencent-Hunyuan","fullname":"Tencent Hunyuan","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/62d22496c58f969c152bcefd/woKSjt2wXvBNKussyYPsa.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"652d3023cdb2a91205709b6a","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/652d3023cdb2a91205709b6a/x-MaRkBwSdZ0xIYDJg09u.jpeg","isPro":false,"fullname":"Hanxun Yu","user":"JonnyYu828","type":"user"},{"_id":"6258a6455ea3a0a9b6de3f22","avatarUrl":"/avatars/6eeed72a97fb24465e5e65583fbe50cf.svg","isPro":false,"fullname":"Lei Ke","user":"lkeab","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"6645f953c39288df638dbdd5","name":"Tencent-Hunyuan","fullname":"Tencent Hunyuan","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/62d22496c58f969c152bcefd/woKSjt2wXvBNKussyYPsa.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.06891.md"}">

Get this paper in your agent:

Don't have the latest CLI?

Cite arxiv.org/abs/2606.06891 in a Space README.md to link it from this page.

Add this paper to a collection to link it from this page.