Tweet: <a href=\"https://x.com/bidhan/status/2082098687464046684\" rel=\"nofollow\">https://x.com/bidhan/status/2082098687464046684</a></p>\n","updatedAt":"2026-07-28T15:13:18.590Z","author":{"_id":"5f1158120c833276f61f1a84","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1608042047613-5f1158120c833276f61f1a84.jpeg","fullname":"Niels Rogge","name":"nielsr","type":"user","isPro":false,"isHf":true,"isHfAdmin":false,"isMod":false,"followerCount":1262,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.5650634169578552},"editors":["nielsr"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1608042047613-5f1158120c833276f61f1a84.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2607.23909","authors":[{"_id":"6a68bc0c5dad3683ba934909","name":"Sen Wang","hidden":false},{"_id":"6a68bc0c5dad3683ba93490a","name":"R. Gnana Praveen","hidden":false},{"_id":"6a68bc0c5dad3683ba93490b","name":"Bidhan Roy","hidden":false},{"_id":"6a68bc0c5dad3683ba93490c","name":"Marcos Villagra","hidden":false}],"publishedAt":"2026-07-27T00:00:00.000Z","submittedOnDailyAt":"2026-07-28T00:00:00.000Z","title":"WorldDiT: A Unified Diffusion Architecture for World and Action Modeling","submittedOnDailyBy":{"_id":"5f1158120c833276f61f1a84","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1608042047613-5f1158120c833276f61f1a84.jpeg","isPro":false,"fullname":"Niels Rogge","user":"nielsr","type":"user","name":"nielsr"},"summary":"Many recent robot policies pursue stronger control by using large pretrained vision-language models (VLMs) as the action backbone. We introduce WorldDiT, a unified diffusion transformer architecture that couples action generation with visual world modeling and achieves strong performance without a large pretrained VLM action backbone. During training, a single diffusion transformer generates continuous action chunks and predicts normalized RGB patch targets from future camera frames. Across four LIBERO simulation suites, WorldDiT lies on the reported Pareto frontier for total model parameters and mean success among methods reporting all four suites. These results provide a strong sub-billion-parameter baseline for future scaling studies.","upvotes":0,"discussionId":"6a68bc0c5dad3683ba93490d","organization":{"_id":"69404a2d504d98ba21c52be1","name":"bageldotcom","fullname":"Bagel Labs","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/6862a65dd478e0d9d375be70/5cu7w3liwUf0OgcntR_rp.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[],"acceptLanguages":["en"],"organization":{"_id":"69404a2d504d98ba21c52be1","name":"bageldotcom","fullname":"Bagel Labs","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/6862a65dd478e0d9d375be70/5cu7w3liwUf0OgcntR_rp.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2607/2607.23909.md","query":{}}">
WorldDiT: A Unified Diffusion Architecture for World and Action Modeling
Abstract
Many recent robot policies pursue stronger control by using large pretrained vision-language models (VLMs) as the action backbone. We introduce WorldDiT, a unified diffusion transformer architecture that couples action generation with visual world modeling and achieves strong performance without a large pretrained VLM action backbone. During training, a single diffusion transformer generates continuous action chunks and predicts normalized RGB patch targets from future camera frames. Across four LIBERO simulation suites, WorldDiT lies on the reported Pareto frontier for total model parameters and mean success among methods reporting all four suites. These results provide a strong sub-billion-parameter baseline for future scaling studies.
Community
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2607.23909 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2607.23909 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.