Hugging Face Daily Papers · July 21, 2026 · 3 min read

HOMIE: Human-object Centric Video Personalization via Multimodal Intelligent Enchancement

Mirrored from Hugging Face Daily Papers for archival readability. Support the source by reading on the original site.

Like Read original ↗

Checkpoint: <a href=\"https://huggingface.co/yychai/homie-r2v-wan2.1\">https://huggingface.co/yychai/homie-r2v-wan2.1</a></p>\n","updatedAt":"2026-07-21T03:44:43.064Z","author":{"_id":"6629d7c9fa14eaccf07d8633","avatarUrl":"/avatars/dceb2f6c804c583adf15a3536c8c995b.svg","fullname":"Nan Chen","name":"CNcreator0331","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":11,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.3608654737472534},"editors":["CNcreator0331"],"editorAvatarUrls":["/avatars/dceb2f6c804c583adf15a3536c8c995b.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2607.18217","authors":[{"_id":"6a5ee3ac4fe5d1d13e84ab77","name":"Yiyang Cai","hidden":false},{"_id":"6a5ee3ac4fe5d1d13e84ab78","name":"Nan Chen","hidden":false},{"_id":"6a5ee3ac4fe5d1d13e84ab79","name":"Rongchang Xie","hidden":false},{"_id":"6a5ee3ac4fe5d1d13e84ab7a","name":"Junwen Pan","hidden":false},{"_id":"6a5ee3ac4fe5d1d13e84ab7b","name":"Chunyang Jiang","hidden":false},{"_id":"6a5ee3ac4fe5d1d13e84ab7c","name":"Cheng Chen","hidden":false},{"_id":"6a5ee3ac4fe5d1d13e84ab7d","name":"Wen Zhou","hidden":false},{"_id":"6a5ee3ac4fe5d1d13e84ab7e","name":"Zhenbang Sun","hidden":false},{"_id":"6a5ee3ac4fe5d1d13e84ab7f","name":"Wei Xue","hidden":false},{"_id":"6a5ee3ac4fe5d1d13e84ab80","name":"Wenhan Luo","hidden":false},{"_id":"6a5ee3ac4fe5d1d13e84ab81","name":"Yike Guo","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/6629d7c9fa14eaccf07d8633/8pEwdIfozfPTxLFvH28sB.mp4"],"publishedAt":"2026-07-20T00:00:00.000Z","submittedOnDailyAt":"2026-07-21T00:00:00.000Z","title":"HOMIE: Human-object Centric Video Personalization via Multimodal Intelligent Enchancement","submittedOnDailyBy":{"_id":"6629d7c9fa14eaccf07d8633","avatarUrl":"/avatars/dceb2f6c804c583adf15a3536c8c995b.svg","isPro":false,"fullname":"Nan Chen","user":"CNcreator0331","type":"user","name":"CNcreator0331"},"summary":"Human-object centric video personalization (HOCVP) is a core task within subject-driven video generation. However, existing methods suffer from two key limitations. First, most approaches focusing on inter-subject personalization still struggle to strike a balance between high subject fidelity and accurate interaction patterns between humans and diverse objects, especially when objects represent abstract concepts such as logos. Second, while intra-subject references (e.g., OCR maps, multi-view inputs) are expected to enhance subject fidelity, most existing works lack mechanisms to understand such latent correspondence. To address both challenges, we propose HOMIE, an HOCVP framework that tackles both inter- and intra-subject input settings in a unified manner. Compared to previous approaches, HOMIE proposes a better MLLM integration strategy to extract knowledge of reference-level relationships without compromising the controllability of text encoders or incurring costly re-alignment. Specifically, we introduce global multimodal guidance within self-attention to better align MLLM-derived semantic features with VAE tokens. Furthermore, we propose modality-reference embedding to differentiate tokens from MLLM features and VAE tokens and associate intra-subject reference image tokens. Extensive experiments validate that our method achieves state-of-the-art performance across various HOCVP tasks. Project Page: https://yiyangcai.github.io/homie-page.github.io/","upvotes":25,"discussionId":"6a5ee3ad4fe5d1d13e84ab82","projectPage":"https://yiyangcai.github.io/homie-page.github.io/","githubRepo":"https://github.com/YIYANGCAI/HOMIE","githubRepoAddedBy":"user","githubStars":11},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6629d7c9fa14eaccf07d8633","avatarUrl":"/avatars/dceb2f6c804c583adf15a3536c8c995b.svg","isPro":false,"fullname":"Nan Chen","user":"CNcreator0331","type":"user"},{"_id":"6513a776584d106e628ddffa","avatarUrl":"/avatars/48fabdf79faf2c5f78402549c41ea119.svg","isPro":false,"fullname":"Yiyang CAI","user":"yychai","type":"user"},{"_id":"64aea8d603246ffd04b11606","avatarUrl":"/avatars/8848373a4974613383b9dd63b809bc2b.svg","isPro":false,"fullname":"Weinan Jia","user":"WeinanJia","type":"user"},{"_id":"64cbc3e38256a8efea569b20","avatarUrl":"/avatars/7f41746c6786308965a2440e9f06dbd1.svg","isPro":false,"fullname":"pengyuan","user":"yuanyaa","type":"user"},{"_id":"6846bc1873f604b8d273e04f","avatarUrl":"/avatars/0110b442f893dfc74e2051dfa2e9aee6.svg","isPro":false,"fullname":"Haozhe Luo","user":"Carloooooo","type":"user"},{"_id":"63edd2d1f765928ceeb49057","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1676530369930-noauth.png","isPro":false,"fullname":"Yaorui SHI","user":"yrshi","type":"user"},{"_id":"666a42e2f5c766c0fb7c7f22","avatarUrl":"/avatars/642ee0fb680880abd8ed1a01e114d52a.svg","isPro":false,"fullname":"Zhangjingyi","user":"21J","type":"user"},{"_id":"6439761abb7ded0a0fefa50e","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6439761abb7ded0a0fefa50e/v2WO5FvJMG3bARZolEJpn.jpeg","isPro":false,"fullname":"Deserts","user":"panjunwen","type":"user"},{"_id":"6315f10a2fd6e930e420c3fd","avatarUrl":"/avatars/b8c7be6d1ef3d2c84e174347d8134d41.svg","isPro":false,"fullname":"rongchang xie","user":"ttttyyyy","type":"user"},{"_id":"6a5eedfaf63c6e2ed06d23a5","avatarUrl":"/avatars/daaacb5fda1b7fd671eeef5b2a5683db.svg","isPro":false,"fullname":"zhe li","user":"lz73933","type":"user"},{"_id":"6661abe4067fa8fb32623c00","avatarUrl":"/avatars/4658bc1ed096f146dd48bb6a4b25fc30.svg","isPro":false,"fullname":"Shuaijia Chen","user":"Paulia12138","type":"user"},{"_id":"64f0b56a87d05e740359279d","avatarUrl":"/avatars/8934a2e2bf4bcbbc3ae9d75e318c80b7.svg","isPro":false,"fullname":"Yiyang Zhang","user":"Anti-clockwise","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2607/2607.18217.md","query":{}}">

Papers

arxiv:2607.18217

HOMIE: Human-object Centric Video Personalization via Multimodal Intelligent Enchancement

Published on Jul 20

· Submitted by

Nan Chen on Jul 21

Upvote

Authors:

Abstract

Human-object centric video personalization (HOCVP) is a core task within subject-driven video generation. However, existing methods suffer from two key limitations. First, most approaches focusing on inter-subject personalization still struggle to strike a balance between high subject fidelity and accurate interaction patterns between humans and diverse objects, especially when objects represent abstract concepts such as logos. Second, while intra-subject references (e.g., OCR maps, multi-view inputs) are expected to enhance subject fidelity, most existing works lack mechanisms to understand such latent correspondence. To address both challenges, we propose HOMIE, an HOCVP framework that tackles both inter- and intra-subject input settings in a unified manner. Compared to previous approaches, HOMIE proposes a better MLLM integration strategy to extract knowledge of reference-level relationships without compromising the controllability of text encoders or incurring costly re-alignment. Specifically, we introduce global multimodal guidance within self-attention to better align MLLM-derived semantic features with VAE tokens. Furthermore, we propose modality-reference embedding to differentiate tokens from MLLM features and VAE tokens and associate intra-subject reference image tokens. Extensive experiments validate that our method achieves state-of-the-art performance across various HOCVP tasks. Project Page: https://yiyangcai.github.io/homie-page.github.io/

View arXiv page View PDF Project page GitHub 11 Add to collection