<video src=\"https://cdn-uploads.huggingface.co/production/uploads/68c9076a8afebb210e6de398/Joey_APKbYL0u3gBuAI4_.mp4\" controls=\"\" class=\"max-w-full!\"></video></p>","updatedAt":"2026-07-24T03:37:24.580Z","author":{"_id":"68c9076a8afebb210e6de398","avatarUrl":"/avatars/e418191eb05523e79a9c2e9edee82aa4.svg","fullname":"tianlezeng","name":"tianlezeng","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.4543112814426422},"editors":["tianlezeng"],"editorAvatarUrls":["/avatars/e418191eb05523e79a9c2e9edee82aa4.svg"],"reactions":[{"reaction":"😎","users":["tianlezeng","JianweiPeng","hill77","hjyeee","guixu-zheng","F0cuS","dannisyang","ziyucherm"],"count":8}],"isReport":false}},{"id":"6a62deb8761996546f5fa64a","author":{"_id":"68c9076a8afebb210e6de398","avatarUrl":"/avatars/e418191eb05523e79a9c2e9edee82aa4.svg","fullname":"tianlezeng","name":"tianlezeng","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false},"createdAt":"2026-07-24T03:40:40.000Z","type":"comment","data":{"edited":true,"hidden":false,"latest":{"raw":"New embodied visual tracking SOTA on EVT Bench🏆\ncode, data and ckpt will be released on Github(https://github.com/MedlarTea/referTrack), welcome to try💪","html":"<p>New embodied visual tracking SOTA on EVT Bench🏆<br>code, data and ckpt will be released on Github(<a href=\"https://github.com/MedlarTea/referTrack\" rel=\"nofollow\">https://github.com/MedlarTea/referTrack</a>), welcome to try💪</p>\n","updatedAt":"2026-07-24T03:41:22.694Z","author":{"_id":"68c9076a8afebb210e6de398","avatarUrl":"/avatars/e418191eb05523e79a9c2e9edee82aa4.svg","fullname":"tianlezeng","name":"tianlezeng","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":1,"identifiedLanguage":{"language":"en","probability":0.8619261384010315},"editors":["tianlezeng"],"editorAvatarUrls":["/avatars/e418191eb05523e79a9c2e9edee82aa4.svg"],"reactions":[{"reaction":"👍","users":["JianweiPeng","hill77","chenxiyuxi","hjyeee","HqiTao","Situ1","F0cuS","dannisyang","mingsun0311","tianlezeng","ziyucherm"],"count":11}],"isReport":false}},{"id":"6a62e226a30c59a6929a1441","author":{"_id":"68075e9a362af0cf1808836a","avatarUrl":"/avatars/e2c8cfdddcde3b14e662d1dd3421bf50.svg","fullname":"JianweiPeng","name":"JianweiPeng","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false},"createdAt":"2026-07-24T03:55:18.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"An outstanding job !!!","html":"<p>An outstanding job !!!</p>\n","updatedAt":"2026-07-24T03:55:18.908Z","author":{"_id":"68075e9a362af0cf1808836a","avatarUrl":"/avatars/e2c8cfdddcde3b14e662d1dd3421bf50.svg","fullname":"JianweiPeng","name":"JianweiPeng","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7581977248191833},"editors":["JianweiPeng"],"editorAvatarUrls":["/avatars/e2c8cfdddcde3b14e662d1dd3421bf50.svg"],"reactions":[{"reaction":"👍","users":["dannisyang","ziyucherm","WiPilbot"],"count":3}],"isReport":false}},{"id":"6a62e256a1ae1453ff840296","author":{"_id":"6a62e190b08b1dd9741fdeaf","avatarUrl":"/avatars/246e677e18e3a3cf38f3a828564d5bfb.svg","fullname":"Zhao","name":"Jieting2000","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false},"createdAt":"2026-07-24T03:56:06.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"impressive ","html":"<p>impressive </p>\n","updatedAt":"2026-07-24T03:56:06.827Z","author":{"_id":"6a62e190b08b1dd9741fdeaf","avatarUrl":"/avatars/246e677e18e3a3cf38f3a828564d5bfb.svg","fullname":"Zhao","name":"Jieting2000","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7542515397071838},"editors":["Jieting2000"],"editorAvatarUrls":["/avatars/246e677e18e3a3cf38f3a828564d5bfb.svg"],"reactions":[{"reaction":"👍","users":["dannisyang"],"count":1}],"isReport":false}},{"id":"6a62fafebab213a591de3faa","author":{"_id":"67de8cbf23c37d4c057b9374","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/-eLEEU7cfnuqLqdFsimlB.png","fullname":"Yu Zhan","name":"hoho-sust","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false},"createdAt":"2026-07-24T05:41:18.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"support🤞🤞🤞","html":"<p>support🤞🤞🤞</p>\n","updatedAt":"2026-07-24T05:41:18.795Z","author":{"_id":"67de8cbf23c37d4c057b9374","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/-eLEEU7cfnuqLqdFsimlB.png","fullname":"Yu Zhan","name":"hoho-sust","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8697835206985474},"editors":["hoho-sust"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/-eLEEU7cfnuqLqdFsimlB.png"],"reactions":[],"isReport":false}},{"id":"6a6313b12e1bc05d60361a3f","author":{"_id":"693a43ed2981a124d3c8b2ec","avatarUrl":"/avatars/da26cd7c399d8b39936e4d9dcc194a73.svg","fullname":"han bao","name":"hill77","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false},"createdAt":"2026-07-24T07:26:41.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"impressive","html":"<p>impressive</p>\n","updatedAt":"2026-07-24T07:26:41.841Z","author":{"_id":"693a43ed2981a124d3c8b2ec","avatarUrl":"/avatars/da26cd7c399d8b39936e4d9dcc194a73.svg","fullname":"han bao","name":"hill77","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7542515397071838},"editors":["hill77"],"editorAvatarUrls":["/avatars/da26cd7c399d8b39936e4d9dcc194a73.svg"],"reactions":[{"reaction":"👍","users":["ziyucherm"],"count":1}],"isReport":false}},{"id":"6a631c45fd7aedc88a08defa","author":{"_id":"6a631aea36d21dc238772cb7","avatarUrl":"/avatars/740122b0b850097c13f9faf5205dc947.svg","fullname":"Yugong Wang","name":"KevinWong216","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false},"createdAt":"2026-07-24T08:03:17.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"brilliant!!","html":"<p>brilliant!!</p>\n","updatedAt":"2026-07-24T08:03:17.703Z","author":{"_id":"6a631aea36d21dc238772cb7","avatarUrl":"/avatars/740122b0b850097c13f9faf5205dc947.svg","fullname":"Yugong Wang","name":"KevinWong216","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"es","probability":0.2808363437652588},"editors":["KevinWong216"],"editorAvatarUrls":["/avatars/740122b0b850097c13f9faf5205dc947.svg"],"reactions":[],"isReport":false}},{"id":"6a632349c7c551d08fad6ca3","author":{"_id":"681c73a8598bbfd2e829f1eb","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/41EaWp6WCA8MyNOmzxCml.png","fullname":"L","name":"ygling","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":4,"isUserFollowing":false},"createdAt":"2026-07-24T08:33:13.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"impressive","html":"<p>impressive</p>\n","updatedAt":"2026-07-24T08:33:13.563Z","author":{"_id":"681c73a8598bbfd2e829f1eb","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/41EaWp6WCA8MyNOmzxCml.png","fullname":"L","name":"ygling","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":4,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7542515397071838},"editors":["ygling"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/41EaWp6WCA8MyNOmzxCml.png"],"reactions":[],"isReport":false}},{"id":"6a6341764f70c8186aaa6c16","author":{"_id":"6a63411206a22e3ca53a0931","avatarUrl":"/avatars/84d19cc7551653a8c751c1e3664fd54d.svg","fullname":"Chaoxin Yang","name":"dannisyang","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false},"createdAt":"2026-07-24T10:41:58.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Really good work👍","html":"<p>Really good work👍</p>\n","updatedAt":"2026-07-24T10:41:58.613Z","author":{"_id":"6a63411206a22e3ca53a0931","avatarUrl":"/avatars/84d19cc7551653a8c751c1e3664fd54d.svg","fullname":"Chaoxin Yang","name":"dannisyang","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9122666716575623},"editors":["dannisyang"],"editorAvatarUrls":["/avatars/84d19cc7551653a8c751c1e3664fd54d.svg"],"reactions":[],"isReport":false}},{"id":"6a634bf47110a7d7d8866a15","author":{"_id":"69e5adb5c0c90b34876f2ce2","avatarUrl":"/avatars/5adb24a2d3824d40058381865980d5ef.svg","fullname":"ming","name":"mingsun0311","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false},"createdAt":"2026-07-24T11:26:44.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Goooooooood Work!","html":"<p>Goooooooood Work!</p>\n","updatedAt":"2026-07-24T11:26:44.478Z","author":{"_id":"69e5adb5c0c90b34876f2ce2","avatarUrl":"/avatars/5adb24a2d3824d40058381865980d5ef.svg","fullname":"ming","name":"mingsun0311","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"hu","probability":0.33349162340164185},"editors":["mingsun0311"],"editorAvatarUrls":["/avatars/5adb24a2d3824d40058381865980d5ef.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2607.20061","authors":[{"_id":"6a61f099c3792c34f5a042a2","name":"Hanjing Ye","hidden":false},{"_id":"6a61f099c3792c34f5a042a3","name":"Tianle Zeng","hidden":false},{"_id":"6a61f099c3792c34f5a042a4","name":"Jiazhao Zhang","hidden":false},{"_id":"6a61f099c3792c34f5a042a5","name":"Shaoan Wang","hidden":false},{"_id":"6a61f099c3792c34f5a042a6","name":"Zibo Zhang","hidden":false},{"_id":"6a61f099c3792c34f5a042a7","name":"Weisi Situ","hidden":false},{"_id":"6a61f099c3792c34f5a042a8","name":"Yuchen Zhou","hidden":false},{"_id":"6a61f099c3792c34f5a042a9","name":"Yonggen Ling","hidden":false},{"_id":"6a61f099c3792c34f5a042aa","name":"Hong Zhang","hidden":false}],"publishedAt":"2026-07-22T00:00:00.000Z","submittedOnDailyAt":"2026-07-24T00:00:00.000Z","title":"ReferTrack: Referring Then Tracking for Embodied Visual Tracking","submittedOnDailyBy":{"_id":"68c9076a8afebb210e6de398","avatarUrl":"/avatars/e418191eb05523e79a9c2e9edee82aa4.svg","isPro":false,"fullname":"tianlezeng","user":"tianlezeng","type":"user","name":"tianlezeng"},"summary":"Embodied visual tracking (EVT) requires a mobile agent to continuously follow a specific target described in natural language using only onboard vision. While recent vision-language-action (VLA) policies unify target identification and trajectory planning, their chain-of-thought (CoT) reasoning often operates in abstract spatial latents that are difficult to supervise and weakly aligned with explicit image-space detections. To address this, we introduce ReferTrack, a referring-then-tracking paradigm that grounds EVT using a single forward-facing camera. Our model first selects the target from an indexed set of bounding boxes, then decodes tracking waypoints conditioned on this image-grounded decision. To preserve target motion cues over time, ReferTrack maintains a sliding-window queue of previously selected bounding boxes, injecting their geometric features into the visual history via temporal-viewpoint-bbox indicator (TVBI) tokens. We further enhance target identification by co-training on a custom Refer-QA dataset. On EVT-Bench, ReferTrack achieves state-of-the-art single-view performance with success rates of 89.4%, 73.3%, and 74.1% on the single-target, distracted, and ambiguity tracking splits, respectively -- matching or even surpassing several multi-camera baselines on identification-heavy tasks. Finally, real-world deployments on legged and humanoid robots validate its robust sim-to-real transfer capabilities. Code is available at https://github.com/MedlarTea/referTrack.","upvotes":43,"discussionId":"6a61f099c3792c34f5a042ab","projectPage":"https://medlartea.github.io/referTrack/","githubRepo":"https://github.com/MedlarTea/referTrack","githubRepoAddedBy":"user","githubStars":18,"organization":{"_id":"66543b6e420092799d2f625c","name":"tencent","fullname":"Tencent","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/5dd96eb166059660ed1ee413/Lp3m-XLpjQGwBItlvn69q.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"67a57b9c58e7bcbbcc9ea5fe","avatarUrl":"/avatars/88879d60cb2130e3b6d8c33b3483e893.svg","isPro":false,"fullname":"Hanjing Ye","user":"hjyeee","type":"user"},{"_id":"68075e9a362af0cf1808836a","avatarUrl":"/avatars/e2c8cfdddcde3b14e662d1dd3421bf50.svg","isPro":false,"fullname":"JianweiPeng","user":"JianweiPeng","type":"user"},{"_id":"68c9076a8afebb210e6de398","avatarUrl":"/avatars/e418191eb05523e79a9c2e9edee82aa4.svg","isPro":false,"fullname":"tianlezeng","user":"tianlezeng","type":"user"},{"_id":"6a5edc609403a998f2380e17","avatarUrl":"/avatars/fba216fd5d30faac9e71379e4f54ba14.svg","isPro":false,"fullname":"YanciWen","user":"WenYanci","type":"user"},{"_id":"6a62e190b08b1dd9741fdeaf","avatarUrl":"/avatars/246e677e18e3a3cf38f3a828564d5bfb.svg","isPro":false,"fullname":"Zhao","user":"Jieting2000","type":"user"},{"_id":"68b55c4d5ceb7eb99e373623","avatarUrl":"/avatars/825dac3dc3a9b5f378207f30d5e7f0f7.svg","isPro":false,"fullname":"艾慕","user":"aimu04","type":"user"},{"_id":"693aa6dd354be706524730f3","avatarUrl":"/avatars/c89cfc89f19a79a9fbdbd40512447d84.svg","isPro":false,"fullname":"peter","user":"lasgient","type":"user"},{"_id":"6a62eb83cfe66430be4e0cc1","avatarUrl":"/avatars/88658eb6961822fd4572cc6fdd5e9f79.svg","isPro":false,"fullname":"LinJintao","user":"040621DBooker","type":"user"},{"_id":"63c1699e40a26dd2db32400d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63c1699e40a26dd2db32400d/3N0-Zp8igv8-52mXAdiiq.jpeg","isPro":false,"fullname":"Chroma","user":"Chroma111","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"68f0d81aa58a0374a8c8a4a2","avatarUrl":"/avatars/3ce1762d3fa620df1496e8aa5181867a.svg","isPro":false,"fullname":"Yuchen2199","user":"Yuchen2199","type":"user"},{"_id":"678f26f802a1c58b5b355f7a","avatarUrl":"/avatars/b834421c0d8b915219773b1c8a2b9d00.svg","isPro":false,"fullname":"Jiayu Tang","user":"chenxiyuxi","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":2,"organization":{"_id":"66543b6e420092799d2f625c","name":"tencent","fullname":"Tencent","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/5dd96eb166059660ed1ee413/Lp3m-XLpjQGwBItlvn69q.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2607/2607.20061.md","query":{}}">
ReferTrack: Referring Then Tracking for Embodied Visual Tracking
Abstract
Embodied visual tracking (EVT) requires a mobile agent to continuously follow a specific target described in natural language using only onboard vision. While recent vision-language-action (VLA) policies unify target identification and trajectory planning, their chain-of-thought (CoT) reasoning often operates in abstract spatial latents that are difficult to supervise and weakly aligned with explicit image-space detections. To address this, we introduce ReferTrack, a referring-then-tracking paradigm that grounds EVT using a single forward-facing camera. Our model first selects the target from an indexed set of bounding boxes, then decodes tracking waypoints conditioned on this image-grounded decision. To preserve target motion cues over time, ReferTrack maintains a sliding-window queue of previously selected bounding boxes, injecting their geometric features into the visual history via temporal-viewpoint-bbox indicator (TVBI) tokens. We further enhance target identification by co-training on a custom Refer-QA dataset. On EVT-Bench, ReferTrack achieves state-of-the-art single-view performance with success rates of 89.4%, 73.3%, and 74.1% on the single-target, distracted, and ambiguity tracking splits, respectively -- matching or even surpassing several multi-camera baselines on identification-heavy tasks. Finally, real-world deployments on legged and humanoid robots validate its robust sim-to-real transfer capabilities. Code is available at https://github.com/MedlarTea/referTrack.
Community
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2607.20061 in a model README.md to link it from this page.
Cite arxiv.org/abs/2607.20061 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2607.20061 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.