Dexterous manipulation is physics intensive and highly sensitive to modeling errors and perception noise, making sim to real transfer prohibitively challenging. Domain randomization (DR) is commonly used to improve the robustness of learned policies for such tasks, but conventional DR randomizes one instance per episode, offering very limited exposure to the variability of real-world dynamics. To this end, propose Domain Randomized Instance Set (DRIS), which represents and propagates a set of randomized instances simultaneously, providing richer approximation of uncertain dynamics and enabling policies to learn actions that account for multiple possible outcomes. Supported by theoretical analysis, we show that DRIS yields more robust policies and alleviates the need for real world fine tuning, even with a modest number of instances (e.g., 10). Demonstrate this on a challenging reactive catching task. Unlike traditional catching setups that use end effectors designed to mechanically stabilize the object (e.g., curved or enclosing surfaces), this system uses a flat plate that offers no passive stabilization, making the task highly sensitive to noise and requiring rapid reactive motions. The learned policies exhibit strong robustness to uncertainties and achieve reliable zero shot sim to real transfer.</p>\n","updatedAt":"2026-05-20T17:58:28.585Z","author":{"_id":"64b8e82aa62c52b252c827fa","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64b8e82aa62c52b252c827fa/Jyk5PHMXCaRlmWy4mT3Bt.jpeg","fullname":"Rajkumar rawal","name":"rajkumarrawal","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":52,"isUserFollowing":false}},"numEdits":1,"identifiedLanguage":{"language":"en","probability":0.9005784392356873},"editors":["rajkumarrawal"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/64b8e82aa62c52b252c827fa/Jyk5PHMXCaRlmWy4mT3Bt.jpeg"],"reactions":[],"isReport":false}},{"id":"6a0e646f15b989c3b729dbb2","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":358,"isUserFollowing":false},"createdAt":"2026-05-21T01:48:31.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [Tac2Real: Reliable and GPU Visuotactile Simulation for Online Reinforcement Learning and Zero-Shot Real-World Deployment](https://huggingface.co/papers/2603.28475) (2026)\n* [Learning Reactive Dexterous Grasping via Hierarchical Task-Space RL Planning and Joint-Space QP Control](https://huggingface.co/papers/2605.03363) (2026)\n* [Learning Task-Invariant Properties via Dreamer: Enabling Efficient Policy Transfer for Quadruped Robots](https://huggingface.co/papers/2604.02911) (2026)\n* [DexSim2Real: Foundation Model-Guided Sim-to-Real Transfer for Generalizable Dexterous Manipulation](https://huggingface.co/papers/2605.05241) (2026)\n* [Toward Hardware-Agnostic Quadrupedal World Models via Morphology Conditioning](https://huggingface.co/papers/2604.08780) (2026)\n* [VOFA: Visual Object Goal Pushing with Force-Adaptive Control for Humanoids](https://huggingface.co/papers/2605.01518) (2026)\n* [Tele-Catch: Adaptive Teleoperation for Dexterous Dynamic 3D Object Catching](https://huggingface.co/papers/2603.28427) (2026)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"<p>This is an automated message from the <a href=\"https://huggingface.co/librarian-bots\">Librarian Bot</a>. I found the following papers similar to this paper. </p>\n<p>The following papers were recommended by the Semantic Scholar API </p>\n<ul>\n<li><a href=\"https://huggingface.co/papers/2603.28475\">Tac2Real: Reliable and GPU Visuotactile Simulation for Online Reinforcement Learning and Zero-Shot Real-World Deployment</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.03363\">Learning Reactive Dexterous Grasping via Hierarchical Task-Space RL Planning and Joint-Space QP Control</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2604.02911\">Learning Task-Invariant Properties via Dreamer: Enabling Efficient Policy Transfer for Quadruped Robots</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.05241\">DexSim2Real: Foundation Model-Guided Sim-to-Real Transfer for Generalizable Dexterous Manipulation</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2604.08780\">Toward Hardware-Agnostic Quadrupedal World Models via Morphology Conditioning</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.01518\">VOFA: Visual Object Goal Pushing with Force-Adaptive Control for Humanoids</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2603.28427\">Tele-Catch: Adaptive Teleoperation for Dexterous Dynamic 3D Object Catching</a> (2026)</li>\n</ul>\n<p> Please give a thumbs up to this comment if you found it helpful!</p>\n<p> If you want recommendations for any Paper on Hugging Face checkout <a href=\"https://huggingface.co/spaces/librarian-bots/recommend_similar_papers\">this</a> Space</p>\n<p> You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: <code><span class=\"SVELTE_PARTIAL_HYDRATER contents\" data-target=\"UserMention\" data-props=\"{"user":"librarian-bot"}\"><span class=\"inline-block\"><span class=\"contents\"><a href=\"/librarian-bot\">@<span class=\"underline\">librarian-bot</span></a></span> </span></span> recommend</code></p>\n","updatedAt":"2026-05-21T01:48:31.756Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":358,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.6873049736022949},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2605.09789","authors":[{"_id":"6a0df012d1ef9ecdf71c0e78","name":"Kejia Ren","hidden":false},{"_id":"6a0df012d1ef9ecdf71c0e79","name":"Gaotian Wang","hidden":false},{"_id":"6a0df012d1ef9ecdf71c0e7a","name":"Andrew S. Morgan","hidden":false},{"_id":"6a0df012d1ef9ecdf71c0e7b","name":"Kaiyu Hang","hidden":false}],"publishedAt":"2026-05-10T00:00:00.000Z","submittedOnDailyAt":"2026-05-20T00:00:00.000Z","title":"Zero-Shot Sim-to-Real Robot Learning: A Dexterous Manipulation Study on Reactive Catching","submittedOnDailyBy":{"_id":"64b8e82aa62c52b252c827fa","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64b8e82aa62c52b252c827fa/Jyk5PHMXCaRlmWy4mT3Bt.jpeg","isPro":true,"fullname":"Rajkumar rawal","user":"rajkumarrawal","type":"user","name":"rajkumarrawal"},"summary":"Dexterous manipulation is physics-intensive and highly sensitive to modeling errors and perception noise, making sim-to-real transfer prohibitively challenging. Domain randomization (DR) is commonly used to improve the robustness of learned policies for such tasks, but conventional DR randomizes one instance per episode, offering very limited exposure to the variability of real-world dynamics. To this end, we propose Domain-Randomized Instance Set (DRIS), which represents and propagates a set of randomized instances simultaneously, providing richer approximation of uncertain dynamics and enabling policies to learn actions that account for multiple possible outcomes. Supported by theoretical analysis, we show that DRIS yields more robust policies and alleviates the need for real-world fine-tuning, even with a modest number of instances (e.g., 10). We demonstrate this on a challenging reactive catching task. Unlike traditional catching setups that use end-effectors designed to mechanically stabilize the object (e.g., curved or enclosing surfaces), our system uses a flat plate that offers no passive stabilization, making the task highly sensitive to noise and requiring rapid reactive motions. The learned policies exhibit strong robustness to uncertainties and achieve reliable zero-shot sim-to-real transfer.","upvotes":1,"discussionId":"6a0df012d1ef9ecdf71c0e7c","ai_summary":"Domain-Randomized Instance Set (DRIS) enables robust policy learning for dexterous manipulation tasks by simultaneously representing multiple randomized instances, achieving strong sim-to-real transfer without extensive real-world fine-tuning.","ai_keywords":["domain randomization","sim-to-real transfer","reactive catching","domain-randomized instance set","policy learning","uncertainty quantification","zero-shot transfer"],"organization":{"_id":"69fbc318a4527472926f3640","name":"Rice-RobotPI-Lab","fullname":"Rice RobotPI Lab","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/6423e59441289875f61d850d/Njw_qBfr_7iLVevQc0jQM.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"64b8e82aa62c52b252c827fa","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64b8e82aa62c52b252c827fa/Jyk5PHMXCaRlmWy4mT3Bt.jpeg","isPro":true,"fullname":"Rajkumar rawal","user":"rajkumarrawal","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"69fbc318a4527472926f3640","name":"Rice-RobotPI-Lab","fullname":"Rice RobotPI Lab","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/6423e59441289875f61d850d/Njw_qBfr_7iLVevQc0jQM.png"}}">
Zero-Shot Sim-to-Real Robot Learning: A Dexterous Manipulation Study on Reactive Catching
Abstract
Domain-Randomized Instance Set (DRIS) enables robust policy learning for dexterous manipulation tasks by simultaneously representing multiple randomized instances, achieving strong sim-to-real transfer without extensive real-world fine-tuning.
AI-generated summary
Dexterous manipulation is physics-intensive and highly sensitive to modeling errors and perception noise, making sim-to-real transfer prohibitively challenging. Domain randomization (DR) is commonly used to improve the robustness of learned policies for such tasks, but conventional DR randomizes one instance per episode, offering very limited exposure to the variability of real-world dynamics. To this end, we propose Domain-Randomized Instance Set (DRIS), which represents and propagates a set of randomized instances simultaneously, providing richer approximation of uncertain dynamics and enabling policies to learn actions that account for multiple possible outcomes. Supported by theoretical analysis, we show that DRIS yields more robust policies and alleviates the need for real-world fine-tuning, even with a modest number of instances (e.g., 10). We demonstrate this on a challenging reactive catching task. Unlike traditional catching setups that use end-effectors designed to mechanically stabilize the object (e.g., curved or enclosing surfaces), our system uses a flat plate that offers no passive stabilization, making the task highly sensitive to noise and requiring rapid reactive motions. The learned policies exhibit strong robustness to uncertainties and achieve reliable zero-shot sim-to-real transfer.
Community
Dexterous manipulation is physics intensive and highly sensitive to modeling errors and perception noise, making sim to real transfer prohibitively challenging. Domain randomization (DR) is commonly used to improve the robustness of learned policies for such tasks, but conventional DR randomizes one instance per episode, offering very limited exposure to the variability of real-world dynamics. To this end, propose Domain Randomized Instance Set (DRIS), which represents and propagates a set of randomized instances simultaneously, providing richer approximation of uncertain dynamics and enabling policies to learn actions that account for multiple possible outcomes. Supported by theoretical analysis, we show that DRIS yields more robust policies and alleviates the need for real world fine tuning, even with a modest number of instances (e.g., 10). Demonstrate this on a challenging reactive catching task. Unlike traditional catching setups that use end effectors designed to mechanically stabilize the object (e.g., curved or enclosing surfaces), this system uses a flat plate that offers no passive stabilization, making the task highly sensitive to noise and requiring rapid reactive motions. The learned policies exhibit strong robustness to uncertainties and achieve reliable zero shot sim to real transfer.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2605.09789 in a model README.md to link it from this page.
Cite arxiv.org/abs/2605.09789 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2605.09789 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.