Inverse rendering of urban scenes from captured videos enables numerous applications, including content creation and autonomous driving simulation. Physically-based rendering methods follow and control lighting physics, but suffer from reconstruction and rendering artifacts. While generative models produce realistic videos, they offer limited consistency and controllability. We present BRDFusion, a unified framework that combines two complementary models for inverse and forward rendering. Specifically, BRDFusion recovers explicit, consistent scene properties with physical modeling and alleviates optimization ambiguity with generative priors. During forward rendering, the physical model provides controllable rendering from the scene configuration, and the generative model denoises and fixes artifacts. Therefore, our method produces high-quality videos while allowing precise control, outperforming baselines in real and synthetic scenes. Moreover, BRDFusion supports novel-view relighting, night simulation, and dynamic object insertion/editing. Project page: <a href=\"https://shigon255.github.io/brdfusion-page/\" rel=\"nofollow\">https://shigon255.github.io/brdfusion-page/</a></p>\n","updatedAt":"2026-06-16T03:18:59.871Z","author":{"_id":"6459d5da3b6fafd9664807ab","avatarUrl":"/avatars/57430d1bbde3a2fe5586e5fbcafb0e74.svg","fullname":"Yu-Lun Liu","name":"yulunliu","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":9,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8549102544784546},"editors":["yulunliu"],"editorAvatarUrls":["/avatars/57430d1bbde3a2fe5586e5fbcafb0e74.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.17049","authors":[{"_id":"6a30c021a0d4daae4285fedd","name":"Yi-Ruei Liu","hidden":false},{"_id":"6a30c021a0d4daae4285fede","user":{"_id":"655f1770f74fa124d1172ec1","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/655f1770f74fa124d1172ec1/bdYocZ1qN50CAfb2z2YLA.png","isPro":false,"fullname":"Jie-Ying Lee","user":"jayinnn","type":"user","name":"jayinnn"},"name":"Jie-Ying Lee","status":"claimed_verified","statusLastChangedAt":"2026-06-16T12:06:56.809Z","hidden":false},{"_id":"6a30c021a0d4daae4285fedf","name":"Zheng-Hui Huang","hidden":false},{"_id":"6a30c021a0d4daae4285fee0","name":"Yu-Lun Liu","hidden":false},{"_id":"6a30c021a0d4daae4285fee1","name":"Chih-Hao Lin","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/6459d5da3b6fafd9664807ab/TNqcrpDr0Vp-7Yx3PVnh8.mp4"],"publishedAt":"2026-06-15T00:00:00.000Z","submittedOnDailyAt":"2026-06-16T00:00:00.000Z","title":"BRDFusion: Physics Meets Generation for Urban Scene Inverse Rendering","submittedOnDailyBy":{"_id":"6459d5da3b6fafd9664807ab","avatarUrl":"/avatars/57430d1bbde3a2fe5586e5fbcafb0e74.svg","isPro":false,"fullname":"Yu-Lun Liu","user":"yulunliu","type":"user","name":"yulunliu"},"summary":"Inverse rendering of urban scenes from captured videos enables numerous applications, including content creation and autonomous driving simulation. Physically-based rendering methods follow and control lighting physics, but suffer from reconstruction and rendering artifacts. While generative models produce realistic videos, they offer limited consistency and controllability. We present BRDFusion, a unified framework that combines two complementary models for inverse and forward rendering. Specifically, BRDFusion recovers explicit, consistent scene properties with physical modeling and alleviates optimization ambiguity with generative priors. During forward rendering, the physical model provides controllable rendering from the scene configuration, and the generative model denoises and fixes artifacts. Therefore, our method produces high-quality videos while allowing precise control, outperforming baselines in real and synthetic scenes. Moreover, BRDFusion supports novel-view relighting, night simulation, and dynamic object insertion/editing. Project page: https://shigon255.github.io/brdfusion-page/","upvotes":22,"discussionId":"6a30c021a0d4daae4285fee2","projectPage":"https://shigon255.github.io/brdfusion-page/","githubRepo":"https://github.com/shigon255/BRDFusion","githubRepoAddedBy":"user","ai_summary":"BRDFusion combines physical modeling and generative priors to achieve high-quality inverse and forward rendering of urban scenes with precise control and artifact reduction.","ai_keywords":["inverse rendering","forward rendering","BRDFusion","physical modeling","generative models","scene properties","optimization ambiguity","controllable rendering","artifact reduction","novel-view relighting","night simulation","dynamic object insertion"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","githubStars":10},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6459d5da3b6fafd9664807ab","avatarUrl":"/avatars/57430d1bbde3a2fe5586e5fbcafb0e74.svg","isPro":false,"fullname":"Yu-Lun Liu","user":"yulunliu","type":"user"},{"_id":"6672fe26c33b5004b69a1d6a","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/Ff8cOS6Y0TPUSihx_hOMe.png","isPro":false,"fullname":"YouZhe","user":"YouZhe","type":"user"},{"_id":"68a41489d9b513a884bca475","avatarUrl":"/avatars/e4caeb16f3c4e7c36835cf26c8cb0d2c.svg","isPro":false,"fullname":"You-Zhe Xie","user":"YouZhe0305","type":"user"},{"_id":"64cdecee2f1f9578a0e701c8","avatarUrl":"/avatars/95a51dd4e1b7b9366ebcbd6028ad148b.svg","isPro":false,"fullname":"Yi-Ruei Liu","user":"Shigon","type":"user"},{"_id":"63484779489226e8b86f1d4d","avatarUrl":"/avatars/a1e703cfbd6574e7fbddfa8e6dfe09f9.svg","isPro":false,"fullname":"Cheng Sun","user":"sunset1995","type":"user"},{"_id":"666a7798b5ad2e4a3fcea70b","avatarUrl":"/avatars/3591e4a480612e8cfe0c87e67087b3bd.svg","isPro":false,"fullname":"許皓翔","user":"Jamichsu","type":"user"},{"_id":"6672ebc506b6d49dda7598c5","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6672ebc506b6d49dda7598c5/9yUeKzZZVtBoy2L-dNPMf.png","isPro":false,"fullname":"Sytwu","user":"Sytwu","type":"user"},{"_id":"666afb91e936f6cbcfc8b50c","avatarUrl":"/avatars/a618c074c9e11e6b9444d0e366efbbdf.svg","isPro":false,"fullname":"LIN, CHIN-YANG","user":"linjohnss","type":"user"},{"_id":"632a340c5f2ff1958c014191","avatarUrl":"/avatars/d51ccb2e2b3e9756dea3d06b6e62f438.svg","isPro":false,"fullname":"Chih-Hao Lin","user":"chih-hao-lin","type":"user"},{"_id":"67173302fd698e5b2a9c91dd","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/7_5xqQEShNkkKpGbjFIjG.png","isPro":false,"fullname":"Bo-Hsu Ke","user":"Hentci","type":"user"},{"_id":"665ebae8bcbb98f60db0b4b1","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/665ebae8bcbb98f60db0b4b1/YTKM4qTZXh_2SeU8U7BfB.webp","isPro":false,"fullname":"Jiale Zhao","user":"Heisenburger2000","type":"user"},{"_id":"66040e5acfb9b90f95a31d76","avatarUrl":"/avatars/bd5b8a4c38aac0531126fd036debb5ed.svg","isPro":false,"fullname":"Brian","user":"Brian9999","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"query":{}}">
BRDFusion: Physics Meets Generation for Urban Scene Inverse Rendering
Abstract
BRDFusion combines physical modeling and generative priors to achieve high-quality inverse and forward rendering of urban scenes with precise control and artifact reduction.
Inverse rendering of urban scenes from captured videos enables numerous applications, including content creation and autonomous driving simulation. Physically-based rendering methods follow and control lighting physics, but suffer from reconstruction and rendering artifacts. While generative models produce realistic videos, they offer limited consistency and controllability. We present BRDFusion, a unified framework that combines two complementary models for inverse and forward rendering. Specifically, BRDFusion recovers explicit, consistent scene properties with physical modeling and alleviates optimization ambiguity with generative priors. During forward rendering, the physical model provides controllable rendering from the scene configuration, and the generative model denoises and fixes artifacts. Therefore, our method produces high-quality videos while allowing precise control, outperforming baselines in real and synthetic scenes. Moreover, BRDFusion supports novel-view relighting, night simulation, and dynamic object insertion/editing. Project page: https://shigon255.github.io/brdfusion-page/
Community
Inverse rendering of urban scenes from captured videos enables numerous applications, including content creation and autonomous driving simulation. Physically-based rendering methods follow and control lighting physics, but suffer from reconstruction and rendering artifacts. While generative models produce realistic videos, they offer limited consistency and controllability. We present BRDFusion, a unified framework that combines two complementary models for inverse and forward rendering. Specifically, BRDFusion recovers explicit, consistent scene properties with physical modeling and alleviates optimization ambiguity with generative priors. During forward rendering, the physical model provides controllable rendering from the scene configuration, and the generative model denoises and fixes artifacts. Therefore, our method produces high-quality videos while allowing precise control, outperforming baselines in real and synthetic scenes. Moreover, BRDFusion supports novel-view relighting, night simulation, and dynamic object insertion/editing. Project page: https://shigon255.github.io/brdfusion-page/
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2606.17049 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.