TD-JEPA keeps the LeWM encoder–predictor and SIGReg backbone, and mines a directed temporal cost from reward-free demonstration logs. Same-trajectory step order supplies positive targets, cross-trajectory pairs act as heuristic negatives, and a rollout-consistency term matches the planner horizon.</p>\n","updatedAt":"2026-07-29T09:10:40.982Z","author":{"_id":"64670b0e0ed2f7a8cba98aa3","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64670b0e0ed2f7a8cba98aa3/mfZlx51fLVRoeC0RyjOvd.jpeg","fullname":"Jiaxin","name":"jbai0318","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8729336261749268},"editors":["jbai0318"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/64670b0e0ed2f7a8cba98aa3/mfZlx51fLVRoeC0RyjOvd.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2607.25337","authors":[{"_id":"6a69c2759d3a1231d492b9f6","name":"Jiaxin Bai","hidden":false},{"_id":"6a69c2759d3a1231d492b9f7","name":"Jiaxuan Xiong","hidden":false}],"publishedAt":"2026-07-28T00:00:00.000Z","submittedOnDailyAt":"2026-07-29T00:00:00.000Z","title":"Temporal-Distance JEPA: Plan-Aware Representation Learning for Latent World Model Predictive Control","submittedOnDailyBy":{"_id":"64670b0e0ed2f7a8cba98aa3","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64670b0e0ed2f7a8cba98aa3/mfZlx51fLVRoeC0RyjOvd.jpeg","isPro":false,"fullname":"Jiaxin","user":"jbai0318","type":"user","name":"jbai0318"},"summary":"Joint-Embedding Predictive Architectures (JEPAs) learn world models by predicting in representation space rather than reconstructing pixels, making them a natural backbone for latent model predictive control from offline demonstration logs. JEPA-style training optimizes short-horizon latent prediction, whereas planning requires a multi-step ranking of imagined futures by goal progress. Prior JEPA planners often inherit that ranking from embedding geometry, typically latent Euclidean distance, which arises as a byproduct of representation learning rather than as a progress cost mined from the logs. We propose temporal-distance JEPA (TD-JEPA), which retains the LeWM encoder--predictor backbone and mines a directed temporal cost from reward-free trajectories: same-trajectory step order supplies positive targets, cross-trajectory pairs act as heuristic negatives, and a rollout-consistency term matches the planner horizon. The mined supervision serves two roles: as the deployed planning cost when progress is topological, and as a representation signal that improves Euclidean planning when contact geometry dominates. Under locked evaluation, deploying the mined cost raises Two-Room success to 100.0% versus LeWM's 97.4%, while shared Euclidean planning on the same temporally trained checkpoint raises OGB-Cube by 14.2 points over LeWM and improves Push-T. Against LeWM and the concurrent RC-aux baseline under locked evaluation, TD-JEPA matches or exceeds both methods on every environment. Ablations show that the directed head, cross-trajectory negatives, and rollout consistency each contribute. TD-JEPA narrows the train--plan gap for JEPA world-model planners by discovering temporal progress structure in offline logs and co-designing cost form with plan-time deployment. Code is available at https://github.com/HKBU-KnowComp/TD-JEPA.","upvotes":1,"discussionId":"6a69c2759d3a1231d492b9f8","githubRepo":"https://github.com/HKBU-KnowComp/TD-JEPA","githubRepoAddedBy":"user","githubStars":1,"organization":{"_id":"6a17e7fd5cefe89a1409a23c","name":"HKBU-KnowComp","fullname":"HKBU Knowledge Computation Lab","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/64670b0e0ed2f7a8cba98aa3/yFMrUnXOVCS8Xi9ru0mdg.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"64670b0e0ed2f7a8cba98aa3","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64670b0e0ed2f7a8cba98aa3/mfZlx51fLVRoeC0RyjOvd.jpeg","isPro":false,"fullname":"Jiaxin","user":"jbai0318","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"6a17e7fd5cefe89a1409a23c","name":"HKBU-KnowComp","fullname":"HKBU Knowledge Computation Lab","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/64670b0e0ed2f7a8cba98aa3/yFMrUnXOVCS8Xi9ru0mdg.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2607/2607.25337.md","query":{}}">
Temporal-Distance JEPA: Plan-Aware Representation Learning for Latent World Model Predictive Control
Published on Jul 28
· Submitted by Jiaxin on Jul 29 Abstract
Joint-Embedding Predictive Architectures (JEPAs) learn world models by predicting in representation space rather than reconstructing pixels, making them a natural backbone for latent model predictive control from offline demonstration logs. JEPA-style training optimizes short-horizon latent prediction, whereas planning requires a multi-step ranking of imagined futures by goal progress. Prior JEPA planners often inherit that ranking from embedding geometry, typically latent Euclidean distance, which arises as a byproduct of representation learning rather than as a progress cost mined from the logs. We propose temporal-distance JEPA (TD-JEPA), which retains the LeWM encoder--predictor backbone and mines a directed temporal cost from reward-free trajectories: same-trajectory step order supplies positive targets, cross-trajectory pairs act as heuristic negatives, and a rollout-consistency term matches the planner horizon. The mined supervision serves two roles: as the deployed planning cost when progress is topological, and as a representation signal that improves Euclidean planning when contact geometry dominates. Under locked evaluation, deploying the mined cost raises Two-Room success to 100.0% versus LeWM's 97.4%, while shared Euclidean planning on the same temporally trained checkpoint raises OGB-Cube by 14.2 points over LeWM and improves Push-T. Against LeWM and the concurrent RC-aux baseline under locked evaluation, TD-JEPA matches or exceeds both methods on every environment. Ablations show that the directed head, cross-trajectory negatives, and rollout consistency each contribute. TD-JEPA narrows the train--plan gap for JEPA world-model planners by discovering temporal progress structure in offline logs and co-designing cost form with plan-time deployment. Code is available at https://github.com/HKBU-KnowComp/TD-JEPA.
Community
TD-JEPA keeps the LeWM encoder–predictor and SIGReg backbone, and mines a directed temporal cost from reward-free demonstration logs. Same-trajectory step order supplies positive targets, cross-trajectory pairs act as heuristic negatives, and a rollout-consistency term matches the planner horizon.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2607.25337 in a model README.md to link it from this page.
Cite arxiv.org/abs/2607.25337 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2607.25337 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.