Echo-Memory is a controlled study that systematically evaluates memory mechanisms in action-conditioned world models to address challenges in long-term scene consistency and object retention.</p>\n","updatedAt":"2026-06-09T03:27:30.617Z","author":{"_id":"6039478ab3ecf716b1a5fd4d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg","fullname":"taesiri","name":"taesiri","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":312,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9063133597373962},"editors":["taesiri"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.09803","authors":[{"_id":"6a2786f26dde1c5ef75bcf9e","name":"Wayne King","hidden":false},{"_id":"6a2786f26dde1c5ef75bcf9f","name":"Zeyue Xue","hidden":false},{"_id":"6a2786f26dde1c5ef75bcfa0","name":"Yuxuan Bian","hidden":false},{"_id":"6a2786f26dde1c5ef75bcfa1","name":"Jie Huang","hidden":false},{"_id":"6a2786f26dde1c5ef75bcfa2","name":"Haoran Li","hidden":false},{"_id":"6a2786f26dde1c5ef75bcfa3","name":"Yaowei Li","hidden":false},{"_id":"6a2786f26dde1c5ef75bcfa4","name":"Yaofeng Su","hidden":false},{"_id":"6a2786f26dde1c5ef75bcfa5","name":"Yuming Li","hidden":false},{"_id":"6a2786f26dde1c5ef75bcfa6","name":"Haoyu Wang","hidden":false},{"_id":"6a2786f26dde1c5ef75bcfa7","name":"Shiyi Zhang","hidden":false},{"_id":"6a2786f26dde1c5ef75bcfa8","name":"Songchun Zhang","hidden":false},{"_id":"6a2786f26dde1c5ef75bcfa9","name":"Yuwei Niu","hidden":false},{"_id":"6a2786f26dde1c5ef75bcfaa","name":"Sihan Xu","hidden":false},{"_id":"6a2786f26dde1c5ef75bcfab","name":"Junhao Zhuang","hidden":false},{"_id":"6a2786f26dde1c5ef75bcfac","name":"Haoyang Huang","hidden":false},{"_id":"6a2786f26dde1c5ef75bcfad","name":"Nan Duan","hidden":false}],"publishedAt":"2026-06-08T00:00:00.000Z","submittedOnDailyAt":"2026-06-09T00:00:00.000Z","title":"Echo-Memory: A Controlled Study of Memory in Action World Models","submittedOnDailyBy":{"_id":"6039478ab3ecf716b1a5fd4d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg","isPro":true,"fullname":"taesiri","user":"taesiri","type":"user","name":"taesiri"},"summary":"We present Echo-Memory, a controlled study of memory mechanisms in action-conditioned world models. These models generate multi-segment videos from a first frame, text prompt, and camera-action sequence, but their central failure is often memory rather than local image synthesis: after the camera leaves and returns, the scene or salient object may silently change. Existing memory designs are hard to compare because gains are entangled with backbone, training, retrieval, and evaluation differences. Echo-Memory fixes the action-to-video interface and varies only how history is stored and read by the generator. Under a shared video diffusion backbone, optimizer, camera-action representation, sampler, and evaluation pipeline, we compare raw context, compression-based memory, spatial summaries with different read-out paths, and state-space recurrence. This matched matrix separates four otherwise conflated axes: capacity, compression, read-out, and recurrence. We also evaluate memory through a three-branch protocol: replay quality, in-domain loop revisit, and open-domain return probes. The branches routinely disagree, showing that replay fidelity is not a sufficient proxy for remembering a world. Three findings follow. Raw context is a strong capacity baseline and improves open-domain return far more than it improves replay metrics. Compactness is not a free substitute for capacity: aggressive spatial and hybrid-compression memories lose the salient evidence needed for return. Finally, block-wise state-space recurrence is the strongest open-domain return mechanism in our matrix, showing that the structure of implicit memory matters as much as the decision to use it. These results provide a compact protocol for studying memory in action world models beyond isolated replay metrics.","upvotes":25,"discussionId":"6a2786f26dde1c5ef75bcfae","projectPage":"https://echo-team-joy-future-academy-jd.github.io/Echo-Memory/","githubRepo":"https://github.com/Echo-Team-Joy-Future-Academy-JD/Echo-Memory","githubRepoAddedBy":"user","ai_summary":"Controlled study of memory mechanisms in action-conditioned world models reveals that memory structure and capacity significantly impact open-domain return performance beyond simple replay fidelity measures.","ai_keywords":["action-conditioned world models","video diffusion backbone","memory mechanisms","context memory","compression-based memory","spatial summaries","state-space recurrence","replay quality","loop revisit","return probes"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","githubStars":78},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"66608add236f958513d21d2e","avatarUrl":"/avatars/53eca0891c98cbb93be899885160a983.svg","isPro":false,"fullname":"Weiyang Jin","user":"Wayne-King","type":"user"},{"_id":"63721f5ada3183d9d53cfe1f","avatarUrl":"/avatars/593c14c907848da7dbc9e5418751bd94.svg","isPro":false,"fullname":"Xue Zeyue","user":"xzyhku","type":"user"},{"_id":"6411c801e872ae3fb1e2c96e","avatarUrl":"/avatars/f8898dc13d700e545eedbbfab1c18353.svg","isPro":true,"fullname":"Franklin","user":"Franklinzhang","type":"user"},{"_id":"6362801380c1a705a6ea54ac","avatarUrl":"/avatars/041ad5abf9be42e336938f51ebb8746c.svg","isPro":false,"fullname":"Yaowei Li","user":"Yw22","type":"user"},{"_id":"646eac510867c99c2d3fde08","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/646eac510867c99c2d3fde08/fvIiW7zj4aTNbp16kTNBA.jpeg","isPro":false,"fullname":"Yaofeng Su","user":"Exploration","type":"user"},{"_id":"63779f2024d97f9f7ec6287b","avatarUrl":"/avatars/6df19a3c7e461db1d5d7c221712ef5f5.svg","isPro":false,"fullname":"Min Zhao","user":"GraceZhao","type":"user"},{"_id":"669a11086ce92cea1816d505","avatarUrl":"/avatars/e0b3100eff892560b0f81605c4ec389f.svg","isPro":false,"fullname":"Zhiyuan Xu","user":"Pixtella","type":"user"},{"_id":"67d8f5785d98e6f9bb7b7279","avatarUrl":"/avatars/e0aaa234339d0e8e9a21a4733008de3c.svg","isPro":false,"fullname":"Andy Cao","user":"HathawayNoa0105","type":"user"},{"_id":"670880950e79a8b46f7ff9dd","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/670880950e79a8b46f7ff9dd/hA1TLhwlQblkFsq8wLrkB.jpeg","isPro":false,"fullname":"Juanxi Tian","user":"Juanxi","type":"user"},{"_id":"67344a21db744d70cb9be933","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Z9Eh-asE3ZISNGXOTzTFQ.png","isPro":false,"fullname":"Haoyu Wang","user":"why986","type":"user"},{"_id":"63e9ae22dd2c4effdd6c2256","avatarUrl":"/avatars/2a1da995a1baab503c0277587791a171.svg","isPro":false,"fullname":"Yifei Yu","user":"yuyifei","type":"user"},{"_id":"6039478ab3ecf716b1a5fd4d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg","isPro":true,"fullname":"taesiri","user":"taesiri","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.09803.md"}">
Echo-Memory: A Controlled Study of Memory in Action World Models
Authors: ,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
Abstract
Controlled study of memory mechanisms in action-conditioned world models reveals that memory structure and capacity significantly impact open-domain return performance beyond simple replay fidelity measures.
We present Echo-Memory, a controlled study of memory mechanisms in action-conditioned world models. These models generate multi-segment videos from a first frame, text prompt, and camera-action sequence, but their central failure is often memory rather than local image synthesis: after the camera leaves and returns, the scene or salient object may silently change. Existing memory designs are hard to compare because gains are entangled with backbone, training, retrieval, and evaluation differences. Echo-Memory fixes the action-to-video interface and varies only how history is stored and read by the generator. Under a shared video diffusion backbone, optimizer, camera-action representation, sampler, and evaluation pipeline, we compare raw context, compression-based memory, spatial summaries with different read-out paths, and state-space recurrence. This matched matrix separates four otherwise conflated axes: capacity, compression, read-out, and recurrence. We also evaluate memory through a three-branch protocol: replay quality, in-domain loop revisit, and open-domain return probes. The branches routinely disagree, showing that replay fidelity is not a sufficient proxy for remembering a world. Three findings follow. Raw context is a strong capacity baseline and improves open-domain return far more than it improves replay metrics. Compactness is not a free substitute for capacity: aggressive spatial and hybrid-compression memories lose the salient evidence needed for return. Finally, block-wise state-space recurrence is the strongest open-domain return mechanism in our matrix, showing that the structure of implicit memory matters as much as the decision to use it. These results provide a compact protocol for studying memory in action world models beyond isolated replay metrics.
Community
Echo-Memory is a controlled study that systematically evaluates memory mechanisms in action-conditioned world models to address challenges in long-term scene consistency and object retention.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2606.09803 in a model README.md to link it from this page.
Cite arxiv.org/abs/2606.09803 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2606.09803 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.