Language models trained on large-scale vision-language data have demonstrated strong potential for embodied agents. Harnessing models through embodied tools use offers a promising alternative to end-to-end vision-language-action systems by combining high-level reasoning with external modules for perception, planning, and control. However, it remains unclear what makes an effective harness for embodied manipulation, and to what extent such a harness can unlock embodied capabilities in a wide range of reasoning models. In this work, we present Guava, a harness framework for embodied tool use developed through systematic exploration of the design space of agent workflows, action spaces, and observation spaces. Our study identifies three key ingredients for effective embodied agents: iterative perception-reasoning-action loops, semantic action abstractions, and multimodal observations. To understand whether these design principles are universal even to small models, we develop an end-to-end training pipeline that distills embodied manipulation capabilities into a 4B open-source model using fewer than 2K trajectories collected entirely in simulation. Experimental results in both simulation and real-world environments show performance comparable to frontier proprietary models while exhibiting strong generalization to unseen objects, novel instructions, and long-horizon tasks. Results suggest that a well-designed harness can serve as a scalable, model-agnostic interface for embodied manipulation, enabling strong emergent embodied capabilities in compact open-source models with minimal training data.</p>\n","updatedAt":"2026-06-18T02:03:39.221Z","author":{"_id":"6534a434e778506c5b1e5be8","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6534a434e778506c5b1e5be8/349SdAnjEdIQJSzWvKfZ4.png","fullname":"Xirui Li","name":"AIcell","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":11,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9321481585502625},"editors":["AIcell"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/6534a434e778506c5b1e5be8/349SdAnjEdIQJSzWvKfZ4.png"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.18363","authors":[{"_id":"6a33519359127a45e2c1c541","name":"Haowen Liu","hidden":false},{"_id":"6a33519359127a45e2c1c542","name":"Xirui Li","hidden":false},{"_id":"6a33519359127a45e2c1c543","name":"Shaoxiong Yao","hidden":false},{"_id":"6a33519359127a45e2c1c544","name":"Peng Shi","hidden":false},{"_id":"6a33519359127a45e2c1c545","name":"Tianyi Zhou","hidden":false},{"_id":"6a33519359127a45e2c1c546","name":"Jia-Bin Huang","hidden":false},{"_id":"6a33519359127a45e2c1c547","name":"Furong Huang","hidden":false},{"_id":"6a33519359127a45e2c1c548","name":"Jiayuan Mao","hidden":false}],"publishedAt":"2026-06-16T00:00:00.000Z","submittedOnDailyAt":"2026-06-18T00:00:00.000Z","title":"Guava: An Effective and Universal Harness for Embodied Manipulation","submittedOnDailyBy":{"_id":"6534a434e778506c5b1e5be8","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6534a434e778506c5b1e5be8/349SdAnjEdIQJSzWvKfZ4.png","isPro":true,"fullname":"Xirui Li","user":"AIcell","type":"user","name":"AIcell"},"summary":"Language models trained on large-scale vision-language data have demonstrated strong potential for embodied agents. Harnessing models through embodied tools use offers a promising alternative to end-to-end vision-language-action systems by combining high-level reasoning with external modules for perception, planning, and control. However, it remains unclear what makes an effective harness for embodied manipulation, and to what extent such a harness can unlock embodied capabilities in a wide range of reasoning models. In this work, we present Guava, a harness framework for embodied tool use developed through systematic exploration of the design space of agent workflows, action spaces, and observation spaces. Our study identifies three key ingredients for effective embodied agents: iterative perception-reasoning-action loops, semantic action abstractions, and multimodal observations. To understand whether these design principles are universal even to small models, we develop an end-to-end training pipeline that distills embodied manipulation capabilities into a 4B open-source model using fewer than 2K trajectories collected entirely in simulation. Experimental results in both simulation and real-world environments show performance comparable to frontier proprietary models while exhibiting strong generalization to unseen objects, novel instructions, and long-horizon tasks. Results suggest that a well-designed harness can serve as a scalable, model-agnostic interface for embodied manipulation, enabling strong emergent embodied capabilities in compact open-source models with minimal training data.","upvotes":20,"discussionId":"6a33519359127a45e2c1c549","projectPage":"https://guava-harness.github.io/","ai_summary":"A harness framework for embodied tool use combines high-level reasoning with external modules, enabling compact models to perform complex manipulation tasks with minimal training data.","ai_keywords":["embodied agents","vision-language models","embodied manipulation","agent workflows","action spaces","observation spaces","iterative perception-reasoning-action loops","semantic action abstractions","multimodal observations","end-to-end training","model distillation","simulation training"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct"},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6534a434e778506c5b1e5be8","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6534a434e778506c5b1e5be8/349SdAnjEdIQJSzWvKfZ4.png","isPro":true,"fullname":"Xirui Li","user":"AIcell","type":"user"},{"_id":"69e4526a59b16b255db64b03","avatarUrl":"/avatars/5f382da70656a008449c12e596998a08.svg","isPro":true,"fullname":"Shaoxiong Yao","user":"nf-ai-workshop","type":"user"},{"_id":"6039478ab3ecf716b1a5fd4d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg","isPro":true,"fullname":"taesiri","user":"taesiri","type":"user"},{"_id":"64a8121e35fab7cd04c30ed0","avatarUrl":"/avatars/48849b84703158772f1022932331b143.svg","isPro":false,"fullname":"Chenrui Fan","user":"Fcr09","type":"user"},{"_id":"6560763e152b659e623865ae","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6560763e152b659e623865ae/cTT2jGnPU_8XMrUTvqZ2h.jpeg","isPro":false,"fullname":"Xiao Liang","user":"MasterVito","type":"user"},{"_id":"65031d01cccc7b28a388c719","avatarUrl":"/avatars/9d8c94b6ab8ad8b4faba3221b7e76053.svg","isPro":false,"fullname":"Ming Li","user":"MingLiiii","type":"user"},{"_id":"658408ba1e065c276dd91186","avatarUrl":"/avatars/a50d8d16635363dddcdd46a7a9f6507a.svg","isPro":false,"fullname":"Ziyue Li","user":"Litzy619","type":"user"},{"_id":"61a7e5da79a402e2ba4bf680","avatarUrl":"/avatars/f64d1e41279f63a9302b285f8dbeab7f.svg","isPro":false,"fullname":"Peng Shi","user":"pengshi","type":"user"},{"_id":"68a533105dc0926bfba4bccd","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/o89W_NbBKDMD9wQLugsRw.png","isPro":false,"fullname":"Constant, Yuan Chen","user":"constannnnt","type":"user"},{"_id":"6a1f7612f7e6c350f97fa9c8","avatarUrl":"/avatars/1f6fd29ca7c35fd2d941c1e32f771f79.svg","isPro":false,"fullname":"Constant, Yuan Chen","user":"constannnt","type":"user"},{"_id":"65cf40ebabb231cb72a58da1","avatarUrl":"/avatars/6889c4d1b5001dfb43045d7748ad0e76.svg","isPro":false,"fullname":"Constant, Yuan Chen","user":"Constannnnnt","type":"user"},{"_id":"5f024bd8ff69163f6f59e775","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1652111948107-5f024bd8ff69163f6f59e775.png","isPro":false,"fullname":"Richard He Bai","user":"richardbaihe","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":2,"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.18363.md","query":{}}">
Guava: An Effective and Universal Harness for Embodied Manipulation
Abstract
A harness framework for embodied tool use combines high-level reasoning with external modules, enabling compact models to perform complex manipulation tasks with minimal training data.
Language models trained on large-scale vision-language data have demonstrated strong potential for embodied agents. Harnessing models through embodied tools use offers a promising alternative to end-to-end vision-language-action systems by combining high-level reasoning with external modules for perception, planning, and control. However, it remains unclear what makes an effective harness for embodied manipulation, and to what extent such a harness can unlock embodied capabilities in a wide range of reasoning models. In this work, we present Guava, a harness framework for embodied tool use developed through systematic exploration of the design space of agent workflows, action spaces, and observation spaces. Our study identifies three key ingredients for effective embodied agents: iterative perception-reasoning-action loops, semantic action abstractions, and multimodal observations. To understand whether these design principles are universal even to small models, we develop an end-to-end training pipeline that distills embodied manipulation capabilities into a 4B open-source model using fewer than 2K trajectories collected entirely in simulation. Experimental results in both simulation and real-world environments show performance comparable to frontier proprietary models while exhibiting strong generalization to unseen objects, novel instructions, and long-horizon tasks. Results suggest that a well-designed harness can serve as a scalable, model-agnostic interface for embodied manipulation, enabling strong emergent embodied capabilities in compact open-source models with minimal training data.
Community
Language models trained on large-scale vision-language data have demonstrated strong potential for embodied agents. Harnessing models through embodied tools use offers a promising alternative to end-to-end vision-language-action systems by combining high-level reasoning with external modules for perception, planning, and control. However, it remains unclear what makes an effective harness for embodied manipulation, and to what extent such a harness can unlock embodied capabilities in a wide range of reasoning models. In this work, we present Guava, a harness framework for embodied tool use developed through systematic exploration of the design space of agent workflows, action spaces, and observation spaces. Our study identifies three key ingredients for effective embodied agents: iterative perception-reasoning-action loops, semantic action abstractions, and multimodal observations. To understand whether these design principles are universal even to small models, we develop an end-to-end training pipeline that distills embodied manipulation capabilities into a 4B open-source model using fewer than 2K trajectories collected entirely in simulation. Experimental results in both simulation and real-world environments show performance comparable to frontier proprietary models while exhibiting strong generalization to unseen objects, novel instructions, and long-horizon tasks. Results suggest that a well-designed harness can serve as a scalable, model-agnostic interface for embodied manipulation, enabling strong emergent embodied capabilities in compact open-source models with minimal training data.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2606.18363 in a model README.md to link it from this page.
Cite arxiv.org/abs/2606.18363 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2606.18363 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.