GRAIL is a digital pipeline that synthesizes humanoid loco-manipulation data from 3D assets and video priors, facilitating sim-to-real training of visual robot policies for complex real-world navigation and manipulation.</p>\n","updatedAt":"2026-06-04T02:13:51.801Z","author":{"_id":"6039478ab3ecf716b1a5fd4d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg","fullname":"taesiri","name":"taesiri","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":310,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8486630320549011},"editors":["taesiri"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.05160","authors":[{"_id":"6a20df2b15100c5272a8464c","name":"Tianyi Xie","hidden":false},{"_id":"6a20df2b15100c5272a8464d","name":"Haotian Zhang","hidden":false},{"_id":"6a20df2b15100c5272a8464e","name":"Jinhyung Park","hidden":false},{"_id":"6a20df2b15100c5272a8464f","name":"Zi Wang","hidden":false},{"_id":"6a20df2b15100c5272a84650","name":"Bowen Wen","hidden":false},{"_id":"6a20df2b15100c5272a84651","name":"Jiefeng Li","hidden":false},{"_id":"6a20df2b15100c5272a84652","name":"Xueting Li","hidden":false},{"_id":"6a20df2b15100c5272a84653","name":"Qingwei Ben","hidden":false},{"_id":"6a20df2b15100c5272a84654","name":"Haoyang Weng","hidden":false},{"_id":"6a20df2b15100c5272a84655","name":"Yufei Ye","hidden":false},{"_id":"6a20df2b15100c5272a84656","name":"David Minor","hidden":false},{"_id":"6a20df2b15100c5272a84657","name":"Tingwu Wang","hidden":false},{"_id":"6a20df2b15100c5272a84658","name":"Chenfanfu Jiang","hidden":false},{"_id":"6a20df2b15100c5272a84659","name":"Sanja Fidler","hidden":false},{"_id":"6a20df2b15100c5272a8465a","name":"Jan Kautz","hidden":false},{"_id":"6a20df2b15100c5272a8465b","name":"Linxi Fan","hidden":false},{"_id":"6a20df2b15100c5272a8465c","name":"Yuke Zhu","hidden":false},{"_id":"6a20df2b15100c5272a8465d","name":"Zhengyi Luo","hidden":false},{"_id":"6a20df2b15100c5272a8465e","name":"Umar Iqbal","hidden":false},{"_id":"6a20df2b15100c5272a8465f","name":"Ye Yuan","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/6039478ab3ecf716b1a5fd4d/pF8chhxWNtHf9E7PzvnbV.mp4"],"publishedAt":"2026-06-03T00:00:00.000Z","submittedOnDailyAt":"2026-06-04T00:00:00.000Z","title":"GRAIL: Generating Humanoid Loco-Manipulation from 3D Assets and Video Priors","submittedOnDailyBy":{"_id":"6039478ab3ecf716b1a5fd4d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg","isPro":true,"fullname":"taesiri","user":"taesiri","type":"user","name":"taesiri"},"summary":"Scaling humanoid loco-manipulation requires robot-compatible demonstrations across diverse objects, whole-body motions, and scene geometries, but teleoperation and motion capture are difficult to scale because each collection depends on physical setups, instrumented actors, and robot operation. We present GRAIL, a digital generation pipeline that remains fully virtual until deployment: it composes 3D assets, simulator-ready scenes, and priors from video foundation models (VFMs) to synthesize interactions without rebuilding physical environments or teleoperating the robot. Rather than reconstructing unconstrained in-the-wild videos, GRAIL starts from fully specified 3D configurations in which object geometry, camera parameters, metric scale, environment depth, and a robot-proportioned character are known before video generation and reused during reconstruction. This privileged setup better conditions 4D recovery, allowing model-based object tracking, human motion estimation, and interaction-aware optimization to reconstruct metric 4D human-object interaction (HOI) trajectories with reduced depth ambiguity and morphology mismatch. We retarget the recovered motions to a humanoid robot and train complementary task-general trackers: an object-aware latent adaptor for manipulation and a scene-aware tracker for terrain traversal. GRAIL produces over 20,000 sequences spanning pick-up, object manipulation, sitting, and terrain traversal. Using only GRAIL-generated data, we train egocentric visual policies through a sim-to-real pipeline and deploy them on a Unitree G1 humanoid, achieving 84\\% real-world success on diverse object pick-up and 90\\% success on stair-climbing.","upvotes":4,"discussionId":"6a20df2b15100c5272a84660","projectPage":"https://research.nvidia.com/labs/dair/grail/","githubRepo":"https://github.com/NVlabs/GRAIL","githubRepoAddedBy":"user","ai_summary":"GRAIL generates diverse humanoid manipulation and locomotion data through 3D asset composition and video foundation models, enabling effective sim-to-real transfer for robot control.","ai_keywords":["video foundation models","4D recovery","human-object interaction","sim-to-real pipeline","egocentric visual policies","humanoid robot","object-aware latent adaptor","scene-aware tracker"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","githubStars":1,"organization":{"_id":"60262b67268c201cdc8b7d43","name":"nvidia","fullname":"NVIDIA","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/65df9200dc3292a8983e5017/Vs5FPVCH-VZBipV3qKTuy.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6039478ab3ecf716b1a5fd4d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg","isPro":true,"fullname":"taesiri","user":"taesiri","type":"user"},{"_id":"69eb03a0b6654b11a5f06df7","avatarUrl":"/avatars/4480e2e9de261928956d33333b733466.svg","isPro":false,"fullname":"Tianyi Xie","user":"XPandora77","type":"user"},{"_id":"6a15f0928642d018701db4c4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/tahvvoPjvq2cNvPRAG-31.png","isPro":false,"fullname":"Xu Jiahao","user":"XUJIh3","type":"user"},{"_id":"66935bdc5489e4f73c76bc7b","avatarUrl":"/avatars/129d1e86bbaf764b507501f4feb177db.svg","isPro":false,"fullname":"Abidoye Aanuoluwapo","user":"Aanuoluwapo65","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"60262b67268c201cdc8b7d43","name":"nvidia","fullname":"NVIDIA","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/65df9200dc3292a8983e5017/Vs5FPVCH-VZBipV3qKTuy.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.05160.md"}">

Get this paper in your agent:

Don't have the latest CLI?

Cite arxiv.org/abs/2606.05160 in a model README.md to link it from this page.

Cite arxiv.org/abs/2606.05160 in a Space README.md to link it from this page.

Add this paper to a collection to link it from this page.