SG-WAM aligns world modeling with action generation by predicting future dynamics in a geometry-aware, policy-derived representation space. The 0.9B model achieves 98.5% on LIBERO and 73.0% on LIBERO-Plus without large-scale embodied pretraining.</p>\n","updatedAt":"2026-08-04T15:30:56.659Z","author":{"_id":"66580fb00328a30516e7f243","avatarUrl":"/avatars/fb7cb7562e3d4bb104ab2f56c2fde566.svg","fullname":"Zhao","name":"Ruiteng","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8082791566848755},"editors":["Ruiteng"],"editorAvatarUrls":["/avatars/fb7cb7562e3d4bb104ab2f56c2fde566.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.01397","authors":[{"_id":"6a715b7aec5082b9f872ce58","user":{"_id":"66580fb00328a30516e7f243","avatarUrl":"/avatars/fb7cb7562e3d4bb104ab2f56c2fde566.svg","isPro":false,"fullname":"Zhao","user":"Ruiteng","type":"user","name":"Ruiteng"},"name":"Ruiteng Zhao","status":"claimed_verified","statusLastChangedAt":"2026-08-04T09:04:35.207Z","hidden":false},{"_id":"6a715b7aec5082b9f872ce59","name":"Zhengshen Zhang","hidden":false},{"_id":"6a715b7aec5082b9f872ce5a","name":"Yue Su","hidden":false},{"_id":"6a715b7aec5082b9f872ce5b","name":"Wenshuo Wang","hidden":false},{"_id":"6a715b7aec5082b9f872ce5c","name":"Jiahui Li","hidden":false},{"_id":"6a715b7aec5082b9f872ce5d","name":"Zhiyuan Yang","hidden":false},{"_id":"6a715b7aec5082b9f872ce5e","name":"Francis E. H. Tay","hidden":false},{"_id":"6a715b7aec5082b9f872ce5f","name":"Marcelo H. Ang Jr.","hidden":false},{"_id":"6a715b7aec5082b9f872ce60","name":"Haiyue Zhu","hidden":false}],"publishedAt":"2026-08-02T00:00:00.000Z","submittedOnDailyAt":"2026-08-04T00:00:00.000Z","title":"SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Space","submittedOnDailyBy":{"_id":"66580fb00328a30516e7f243","avatarUrl":"/avatars/fb7cb7562e3d4bb104ab2f56c2fde566.svg","isPro":false,"fullname":"Zhao","user":"Ruiteng","type":"user","name":"Ruiteng"},"summary":"World Action Models (WAMs) couple action generation with prediction of future states. Their effectiveness depends on whether future dynamics are modeled in a space that is both aligned with action generation and sufficiently geometry-aware to capture where and how actions change the scene. Existing WAMs typically satisfy only part of this requirement, relying on either perceptually heavy observation-space targets or auxiliary latent spaces that are not jointly structured for action relevance and geometry. We propose SG-WAM, a self-guided framework that learns geometry-aware action-conditioned dynamics directly in the policy-derived representation space. SG-WAM introduces learnable dynamics tokens and a Self-Guided World Predictor that forecasts their future latent states conditioned on intervening robot actions. Prediction targets are generated by an exponential moving average copy of the same policy backbone, providing stable supervision within the representation family used by the action expert. Geometric supervision further structures the policy image-token representations, providing spatially grounded context for the dynamics tokens and yielding a future-alignment space that is both action-relevant and geometry-aware. Latent future prediction, geometric grounding, and flow-matching action generation are jointly optimized end-to-end in a unified framework. Built on a 0.9B model without large-scale embodied pretraining, SG-WAM achieves 98.5% average success on LIBERO and 73% on LIBERO-Plus, while outperforming strong baselines in both in-distribution and out-of-distribution real-world evaluations.","upvotes":1,"discussionId":"6a715b7aec5082b9f872ce61","projectPage":"https://sg-wam.github.io/"},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"66580fb00328a30516e7f243","avatarUrl":"/avatars/fb7cb7562e3d4bb104ab2f56c2fde566.svg","isPro":false,"fullname":"Zhao","user":"Ruiteng","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.01397.md","query":{}}">
SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Space
Published on Aug 2
· Submitted by Zhao on Aug 4 Abstract
World Action Models (WAMs) couple action generation with prediction of future states. Their effectiveness depends on whether future dynamics are modeled in a space that is both aligned with action generation and sufficiently geometry-aware to capture where and how actions change the scene. Existing WAMs typically satisfy only part of this requirement, relying on either perceptually heavy observation-space targets or auxiliary latent spaces that are not jointly structured for action relevance and geometry. We propose SG-WAM, a self-guided framework that learns geometry-aware action-conditioned dynamics directly in the policy-derived representation space. SG-WAM introduces learnable dynamics tokens and a Self-Guided World Predictor that forecasts their future latent states conditioned on intervening robot actions. Prediction targets are generated by an exponential moving average copy of the same policy backbone, providing stable supervision within the representation family used by the action expert. Geometric supervision further structures the policy image-token representations, providing spatially grounded context for the dynamics tokens and yielding a future-alignment space that is both action-relevant and geometry-aware. Latent future prediction, geometric grounding, and flow-matching action generation are jointly optimized end-to-end in a unified framework. Built on a 0.9B model without large-scale embodied pretraining, SG-WAM achieves 98.5% average success on LIBERO and 73% on LIBERO-Plus, while outperforming strong baselines in both in-distribution and out-of-distribution real-world evaluations.
Community
SG-WAM aligns world modeling with action generation by predicting future dynamics in a geometry-aware, policy-derived representation space. The 0.9B model achieves 98.5% on LIBERO and 73.0% on LIBERO-Plus without large-scale embodied pretraining.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2608.01397 in a model README.md to link it from this page.
Cite arxiv.org/abs/2608.01397 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2608.01397 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.