Hide-and-Seek reformulates VLA failure detection as a coarsely supervised problem, using inter- and intra-trajectory contrastive objectives to induce temporally structured failure signals from trajectory-level labels only.</p>\n","updatedAt":"2026-06-01T01:44:36.696Z","author":{"_id":"6696b81167c22a79a15ebaef","avatarUrl":"/avatars/57ce0329c4a2c46481818bc99c1d7f17.svg","fullname":"Seongheon Park","name":"sam121796","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8422313928604126},"editors":["sam121796"],"editorAvatarUrls":["/avatars/57ce0329c4a2c46481818bc99c1d7f17.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2605.30834","authors":[{"_id":"6a1ce1d5808ddbc3c7d433b2","name":"Seongheon Park","hidden":false},{"_id":"6a1ce1d5808ddbc3c7d433b3","name":"Wendi Li","hidden":false},{"_id":"6a1ce1d5808ddbc3c7d433b4","name":"Changdae Oh","hidden":false},{"_id":"6a1ce1d5808ddbc3c7d433b5","name":"Samuel Yeh","hidden":false},{"_id":"6a1ce1d5808ddbc3c7d433b6","name":"Zsolt Kira","hidden":false},{"_id":"6a1ce1d5808ddbc3c7d433b7","name":"Michael Hagenow","hidden":false},{"_id":"6a1ce1d5808ddbc3c7d433b8","name":"Sharon Li","hidden":false}],"publishedAt":"2026-05-29T00:00:00.000Z","submittedOnDailyAt":"2026-06-01T00:00:00.000Z","title":"Hide-and-Seek in Trajectories: Discovering Failure Signals for VLA Runtime Monitoring","submittedOnDailyBy":{"_id":"6696b81167c22a79a15ebaef","avatarUrl":"/avatars/57ce0329c4a2c46481818bc99c1d7f17.svg","isPro":false,"fullname":"Seongheon Park","user":"sam121796","type":"user","name":"sam121796"},"summary":"Vision-Language-Action (VLA) models enable robots to follow natural language instructions and generalize across diverse tasks, but they remain vulnerable to execution failures that compromise reliability in real-world deployment. Detecting such failures during execution is therefore critical for the robust deployment of embodied systems. Existing failure detection methods either rely on expensive action resampling or external models, while alternatives propagate trajectory-level labels uniformly across every timestep, obscuring localized failure signals. In this paper, we propose Hide-and-Seek, a framework that formulates VLA failure detection as a coarsely supervised learning problem. By combining inter-trajectory and intra-trajectory contrastive objectives, Hide-and-Seek localizes failure-indicative actions and induces temporally structured failure signals from trajectory-level supervision alone, without any step-level annotation. We evaluate Hide-and-Seek on LIBERO, VLABench, and a real-world robotic platform across three representative VLA policies: OpenVLA, π_0, and π_{0.5}.Our method achieves state-of-the-art multi-task failure detection performance with a practical accuracy--timeliness trade-off under conformal prediction, and generalizes well to both seen and unseen tasks.","upvotes":5,"discussionId":"6a1ce1d5808ddbc3c7d433b9","projectPage":"https://seongheon-96.github.io/hide_and_seek_site/","ai_summary":"Hide-and-Seek framework detects robot execution failures in vision-language-action models by localizing failure-indicative actions through contrastive learning from trajectory-level supervision without step-level annotations.","ai_keywords":["Vision-Language-Action models","failure detection","coarsely supervised learning","inter-trajectory contrastive objectives","intra-trajectory contrastive objectives","temporal structure","conformal prediction","OpenVLA","π₀","π₀.₅"],"organization":{"_id":"6279cd50c20d41b28913755e","name":"Uwmadison","fullname":"University of Wisconsin-Madison","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/9Jg5WNG52u9RTFzOgQsII.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6696b81167c22a79a15ebaef","avatarUrl":"/avatars/57ce0329c4a2c46481818bc99c1d7f17.svg","isPro":false,"fullname":"Seongheon Park","user":"sam121796","type":"user"},{"_id":"64bcc06fb567ae97c3272d3d","avatarUrl":"/avatars/bcb61fe9e575154d84913a1501971f1a.svg","isPro":false,"fullname":"Dahye Kim","user":"dahyekim","type":"user"},{"_id":"672fc8ede7c89e44c9757259","avatarUrl":"/avatars/caa0d0de519ea96992c81328c89b3843.svg","isPro":false,"fullname":"Changdae Oh","user":"changdae","type":"user"},{"_id":"63c07f198d1175e3399d2161","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1673559768829-noauth.jpeg","isPro":false,"fullname":"Min-Hsuan Yeh","user":"samuelyeh","type":"user"},{"_id":"64a53edb1adc23da725f3afd","avatarUrl":"/avatars/d3cd53862f0b11fe2d532653062ac7cf.svg","isPro":false,"fullname":"jiapengyue","user":"Jia-py","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"6279cd50c20d41b28913755e","name":"Uwmadison","fullname":"University of Wisconsin-Madison","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/9Jg5WNG52u9RTFzOgQsII.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2605/2605.30834.md"}">
Hide-and-Seek in Trajectories: Discovering Failure Signals for VLA Runtime Monitoring
Abstract
Hide-and-Seek framework detects robot execution failures in vision-language-action models by localizing failure-indicative actions through contrastive learning from trajectory-level supervision without step-level annotations.
AI-generated summary
Vision-Language-Action (VLA) models enable robots to follow natural language instructions and generalize across diverse tasks, but they remain vulnerable to execution failures that compromise reliability in real-world deployment. Detecting such failures during execution is therefore critical for the robust deployment of embodied systems. Existing failure detection methods either rely on expensive action resampling or external models, while alternatives propagate trajectory-level labels uniformly across every timestep, obscuring localized failure signals. In this paper, we propose Hide-and-Seek, a framework that formulates VLA failure detection as a coarsely supervised learning problem. By combining inter-trajectory and intra-trajectory contrastive objectives, Hide-and-Seek localizes failure-indicative actions and induces temporally structured failure signals from trajectory-level supervision alone, without any step-level annotation. We evaluate Hide-and-Seek on LIBERO, VLABench, and a real-world robotic platform across three representative VLA policies: OpenVLA, π_0, and π_{0.5}.Our method achieves state-of-the-art multi-task failure detection performance with a practical accuracy--timeliness trade-off under conformal prediction, and generalizes well to both seen and unseen tasks.
Community
Hide-and-Seek reformulates VLA failure detection as a coarsely supervised problem, using inter- and intra-trajectory contrastive objectives to induce temporally structured failure signals from trajectory-level labels only.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2605.30834 in a model README.md to link it from this page.
Cite arxiv.org/abs/2605.30834 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2605.30834 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.