Abstract:</p>\n<p>Reconstructing complete dynamic objects from monocular video requires integrating visual cues from direct observations with data-driven priors over geometry and appearance. Prior approaches either learn to directly predict per-frame 3D representations from visual input or initialize a 3D representation that is subsequently deformed and refined based on video evidence. However, the former are constrained by the scarcity of 4D training data, while the latter leverage priors only for the initial reconstruction and rely solely on video supervision thereafter; neither handles complex in-the-wild scenarios with large deformations and occlusions well.</p>\n<p>We present Lift4D, a test-time optimization framework that addresses both limitations. First, we adapt an existing single-view 3D reconstruction model to yield temporally consistent per-frame predictions via causal latent conditioning, providing a coherent initialization for a deformable 3D Gaussian Splatting representation. We then “sculpt” this representation to match the input video through an occlusion-aware optimization that faithfully recovers visible surface details while completing unobserved regions using a view-conditioned diffusion prior. We demonstrate that Lift4D clearly improves over prior 4D reconstruction methods, particularly on challenging in-the-wild sequences with severe occlusions and non-rigid motion.</p>\n","updatedAt":"2026-06-23T20:40:22.486Z","author":{"_id":"646ff038799a974be31bb344","avatarUrl":"/avatars/d9dc17246fba8360e709235f55445ef5.svg","fullname":"Yehonathan Litman","name":"thebluser","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8692598938941956},"editors":["thebluser"],"editorAvatarUrls":["/avatars/d9dc17246fba8360e709235f55445ef5.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.23688","authors":[{"_id":"6a3acc550a86ac3098d5d51e","name":"Yehonathan Litman","hidden":false},{"_id":"6a3acc550a86ac3098d5d51f","name":"Xiaoxuan Ma","hidden":false},{"_id":"6a3acc550a86ac3098d5d520","name":"Manan Shah","hidden":false},{"_id":"6a3acc550a86ac3098d5d521","name":"Nicolas Ugrinovic","hidden":false},{"_id":"6a3acc550a86ac3098d5d522","name":"Kris Kitani","hidden":false},{"_id":"6a3acc550a86ac3098d5d523","name":"Fernando De la Torre","hidden":false},{"_id":"6a3acc550a86ac3098d5d524","name":"Shubham Tulsiani","hidden":false}],"publishedAt":"2026-06-22T00:00:00.000Z","submittedOnDailyAt":"2026-06-23T00:00:00.000Z","title":"Lift4D: Harmonizing Single-View 3D Estimation for 4D Reconstruction In-the-Wild","submittedOnDailyBy":{"_id":"646ff038799a974be31bb344","avatarUrl":"/avatars/d9dc17246fba8360e709235f55445ef5.svg","isPro":false,"fullname":"Yehonathan Litman","user":"thebluser","type":"user","name":"thebluser"},"summary":"Reconstructing dynamic non-rigid objects from monocular video requires integrating visual cues from direct observations with data-driven priors over geometry and appearance. Prior approaches either learn to directly predict 4D representations from visual input or initialize a 3D representation that is subsequently deformed and refined based on video evidence. However, the former are constrained by the scarcity of 4D training data, while the latter leverage priors only for the initial reconstruction and rely solely on video supervision thereafter; neither handles complex in-the-wild scenarios with large deformations and occlusions well. We present Lift4D, a test-time optimization framework that addresses both limitations. First, we adapt an existing single-view 3D reconstruction model to yield temporally consistent per-frame predictions via causal latent conditioning, providing a coherent initialization for a deformable 3D Gaussian Splatting representation. We then ``sculpt'' this representation to match the input video through an occlusion-aware optimization that faithfully recovers visible surface details while completing unobserved regions using a view-conditioned diffusion prior. We demonstrate that Lift4D clearly improves over prior 4D reconstruction methods, particularly on challenging in-the-wild sequences with severe occlusions and non-rigid motion.","upvotes":2,"discussionId":"6a3acc550a86ac3098d5d525","projectPage":"https://lift4d.github.io/","githubRepo":"https://github.com/yehonathanlitman/Lift4D","githubRepoAddedBy":"user","ai_summary":"Lift4D presents a test-time optimization framework that combines temporal consistency from single-view 3D reconstruction with deformable 3D Gaussian Splatting and view-conditioned diffusion priors to reconstruct dynamic non-rigid objects from monocular video.","ai_keywords":["monocular video","4D reconstruction","visual cues","data-driven priors","geometry","appearance","3D Gaussian Splatting","test-time optimization","causal latent conditioning","occlusion-aware optimization","view-conditioned diffusion prior"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","githubStars":54},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"646ff038799a974be31bb344","avatarUrl":"/avatars/d9dc17246fba8360e709235f55445ef5.svg","isPro":false,"fullname":"Yehonathan Litman","user":"thebluser","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.23688.md","query":{}}">
Lift4D: Harmonizing Single-View 3D Estimation for 4D Reconstruction In-the-Wild
Abstract
Lift4D presents a test-time optimization framework that combines temporal consistency from single-view 3D reconstruction with deformable 3D Gaussian Splatting and view-conditioned diffusion priors to reconstruct dynamic non-rigid objects from monocular video.
Reconstructing dynamic non-rigid objects from monocular video requires integrating visual cues from direct observations with data-driven priors over geometry and appearance. Prior approaches either learn to directly predict 4D representations from visual input or initialize a 3D representation that is subsequently deformed and refined based on video evidence. However, the former are constrained by the scarcity of 4D training data, while the latter leverage priors only for the initial reconstruction and rely solely on video supervision thereafter; neither handles complex in-the-wild scenarios with large deformations and occlusions well. We present Lift4D, a test-time optimization framework that addresses both limitations. First, we adapt an existing single-view 3D reconstruction model to yield temporally consistent per-frame predictions via causal latent conditioning, providing a coherent initialization for a deformable 3D Gaussian Splatting representation. We then ``sculpt'' this representation to match the input video through an occlusion-aware optimization that faithfully recovers visible surface details while completing unobserved regions using a view-conditioned diffusion prior. We demonstrate that Lift4D clearly improves over prior 4D reconstruction methods, particularly on challenging in-the-wild sequences with severe occlusions and non-rigid motion.
Community
Abstract:
Reconstructing complete dynamic objects from monocular video requires integrating visual cues from direct observations with data-driven priors over geometry and appearance. Prior approaches either learn to directly predict per-frame 3D representations from visual input or initialize a 3D representation that is subsequently deformed and refined based on video evidence. However, the former are constrained by the scarcity of 4D training data, while the latter leverage priors only for the initial reconstruction and rely solely on video supervision thereafter; neither handles complex in-the-wild scenarios with large deformations and occlusions well.
We present Lift4D, a test-time optimization framework that addresses both limitations. First, we adapt an existing single-view 3D reconstruction model to yield temporally consistent per-frame predictions via causal latent conditioning, providing a coherent initialization for a deformable 3D Gaussian Splatting representation. We then “sculpt” this representation to match the input video through an occlusion-aware optimization that faithfully recovers visible surface details while completing unobserved regions using a view-conditioned diffusion prior. We demonstrate that Lift4D clearly improves over prior 4D reconstruction methods, particularly on challenging in-the-wild sequences with severe occlusions and non-rigid motion.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2606.23688 in a model README.md to link it from this page.
Cite arxiv.org/abs/2606.23688 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2606.23688 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.