Developing unified video generation and editing models capable of interpreting interleaved multimodal inputs is a promising yet challenging frontier field. Existing unified frameworks predominantly rely on massive models (typically 13B parameters or more) and incorporate source video conditions for editing by concatenating sequence tokens. This concatenation inevitably doubles the sequence length, quadrupling the computational complexity of the self-attention mechanism and introducing prohibitive overhead. To address these bottlenecks, we present LoomVideo, a highly efficient 5B-parameter unified architecture for both video generation and editing. LoomVideo replaces the standard text encoder with a Multimodal Large Language Model (MLLM) and employs a Deepstack injection mechanism to align multi-layer MLLM features with the Diffusion Transformer (DiT). Crucially, we introduce a zero-overhead Scale-and-Add conditioning approach for video editing. By scaling and directly adding the clean source video latent to the noised target latent, this elegant design eliminates the need for token concatenation, drastically reducing computational cost while maintaining robust capabilities for complex, non-rigid edits. Furthermore, a Negative Temporal RoPE strategy is seamlessly integrated to handle multiple reference images. Extensive experiments demonstrate that our compact 5B model achieves state-of-the-art or highly competitive performance across comprehensive benchmarks, exhibiting exceptional superiority in e-commerce and fashion generation scenarios. Benefiting from the zero-overhead conditioning mechanism, LoomVideo achieves at least a 5.41× acceleration in inference speed compared to models of similar capabilities, paving the way for highly practical and efficient video foundation models.</p>\n","updatedAt":"2026-06-05T02:41:32.457Z","author":{"_id":"657a6eed1ccc3c2a5ea7b585","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/RIQIF-JJdNI0SwJEq_9z7.jpeg","fullname":"Jianzong Wu","name":"jianzongwu","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":15,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.815454363822937},"editors":["jianzongwu"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/RIQIF-JJdNI0SwJEq_9z7.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.06042","authors":[{"_id":"6a222c2d3490a593e87b13c5","name":"Jianzong Wu","hidden":false},{"_id":"6a222c2d3490a593e87b13c6","name":"Hao Lian","hidden":false},{"_id":"6a222c2d3490a593e87b13c7","name":"Jiongfan Yang","hidden":false},{"_id":"6a222c2d3490a593e87b13c8","name":"Dachao Hao","hidden":false},{"_id":"6a222c2d3490a593e87b13c9","name":"Ye Tian","hidden":false},{"_id":"6a222c2d3490a593e87b13ca","name":"Yunhai Tong","hidden":false},{"_id":"6a222c2d3490a593e87b13cb","name":"Jingyuan Zhu","hidden":false},{"_id":"6a222c2d3490a593e87b13cc","name":"Biaolong Chen","hidden":false},{"_id":"6a222c2d3490a593e87b13cd","name":"Qiaosong Qi","hidden":false},{"_id":"6a222c2d3490a593e87b13ce","name":"Aixi Zhang","hidden":false},{"_id":"6a222c2d3490a593e87b13cf","name":"Wanggui He","hidden":false},{"_id":"6a222c2d3490a593e87b13d0","name":"Mushui Liu","hidden":false},{"_id":"6a222c2d3490a593e87b13d1","name":"Jinlong Liu","hidden":false},{"_id":"6a222c2d3490a593e87b13d2","name":"Hao Jiang","hidden":false}],"publishedAt":"2026-06-04T00:00:00.000Z","submittedOnDailyAt":"2026-06-05T00:00:00.000Z","title":"LoomVideo: Unifying Multimodal Inputs into Video Generation and Editing","submittedOnDailyBy":{"_id":"657a6eed1ccc3c2a5ea7b585","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/RIQIF-JJdNI0SwJEq_9z7.jpeg","isPro":false,"fullname":"Jianzong Wu","user":"jianzongwu","type":"user","name":"jianzongwu"},"summary":"Developing unified video generation and editing models capable of interpreting interleaved multimodal inputs is a promising yet challenging frontier field. Existing unified frameworks predominantly rely on massive models (typically 13B parameters or more) and incorporate source video conditions for editing by concatenating sequence tokens. This concatenation inevitably doubles the sequence length, quadrupling the computational complexity of the self-attention mechanism and introducing prohibitive overhead. To address these bottlenecks, we present LoomVideo, a highly efficient 5B-parameter unified architecture for both video generation and editing. LoomVideo replaces the standard text encoder with a Multimodal Large Language Model (MLLM) and employs Deepstack injection mechanism to align multi-layer MLLM features with the Diffusion Transformer (DiT). Crucially, we introduce a zero-overhead Scale-and-Add conditioning approach for video editing. By scaling and directly adding the clean source video latent to the noised target latent, this elegant design eliminates the need for token concatenation, drastically reducing computational cost while maintaining robust capabilities for complex, non-rigid edits. Furthermore, a Negative Temporal RoPE strategy is seamlessly integrated to handle multiple reference images. Extensive experiments demonstrate that our compact 5B model achieves state-of-the-art or highly competitive performance across comprehensive benchmarks, exhibiting exceptional superiority in e-commerce and fashion generation scenarios. Benefiting from the zero-overhead conditioning mechanism, LoomVideo achieves at least a 5.41x acceleration in inference speed compared to models of similar capabilities, paving the way for highly practical and efficient video foundation models.","upvotes":16,"discussionId":"6a222c2d3490a593e87b13d3","projectPage":"https://msalab-pku.github.io/projects/LoomVideo/index.html","githubRepo":"https://github.com/MSALab-PKU/LoomVideo","githubRepoAddedBy":"user","ai_summary":"LoomVideo presents an efficient 5B-parameter unified architecture for video generation and editing that reduces computational overhead through novel conditioning mechanisms and multi-modal alignment techniques.","ai_keywords":["multimodal inputs","diffusion transformer","deepstack injection mechanism","scale-and-add conditioning","negative temporal RoPE","video generation","video editing","multimodal large language model","self-attention mechanism","sequence tokens"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","githubStars":3,"organization":{"_id":"61dcd8e344f59573371b5cb6","name":"PekingUniversity","fullname":"Peking University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/vavgrBsnkSejriUF4lXDE.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"657a6eed1ccc3c2a5ea7b585","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/RIQIF-JJdNI0SwJEq_9z7.jpeg","isPro":false,"fullname":"Jianzong Wu","user":"jianzongwu","type":"user"},{"_id":"6a1e4becfaa0b23a50071f9f","avatarUrl":"/avatars/bd60b8f90a61f2278b66a01431637e2e.svg","isPro":false,"fullname":"MSALab@PKU","user":"MSALab","type":"user"},{"_id":"68a83b76039e9c7b908b6467","avatarUrl":"/avatars/0db215d62de2665ba61e847a92344b5e.svg","isPro":false,"fullname":"Hao Lian","user":"Firefly-lh","type":"user"},{"_id":"6317391acd0254f8026ca3b7","avatarUrl":"/avatars/75f89fc0e8fa284f0456925c105f2660.svg","isPro":false,"fullname":"Qiaosong Qi","user":"kuqs","type":"user"},{"_id":"64e357dd825f4133e7427bf8","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64e357dd825f4133e7427bf8/HwaWhINrzkbXG6SHG2oyf.jpeg","isPro":false,"fullname":"tyfeld","user":"tyfeld","type":"user"},{"_id":"64e2d6ebd5e75575cd5336d2","avatarUrl":"/avatars/4036ca9e5cfb490c83ce1424513a94bc.svg","isPro":false,"fullname":"chenbiaolong","user":"chenbiaolong","type":"user"},{"_id":"65519eb532f278f503b3b2c3","avatarUrl":"/avatars/2e180f7b20189cd2d8a75e05c2913c5d.svg","isPro":false,"fullname":"lalala","user":"QuanjianSong","type":"user"},{"_id":"68370f225b8acaa67dfe76ff","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/68370f225b8acaa67dfe76ff/iNLiNEsEqR6R2i-epe9Nq.jpeg","isPro":false,"fullname":"杨炅凡","user":"Gokottaw434","type":"user"},{"_id":"65bce64b8467e2a3d6a450af","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65bce64b8467e2a3d6a450af/IFjoy4GYMA6oUgYyJfZ1F.jpeg","isPro":false,"fullname":"Chao Tang","user":"Tangc03","type":"user"},{"_id":"65e954e3588ff8b17899f05d","avatarUrl":"/avatars/44b5f4f12102b19d2bb374ad838ab6de.svg","isPro":false,"fullname":"L","user":"Shui-VL","type":"user"},{"_id":"66dbe9343b6063ac837a0aa5","avatarUrl":"/avatars/b44e04e00ea863efc133eaab545962d5.svg","isPro":false,"fullname":"oooooo","user":"kzfkzf","type":"user"},{"_id":"63958b4414513eaf9029ebf1","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/U1g5H071pWRswGAG9UTpo.png","isPro":false,"fullname":"Xiangtai Li","user":"LXT","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"61dcd8e344f59573371b5cb6","name":"PekingUniversity","fullname":"Peking University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/vavgrBsnkSejriUF4lXDE.png"}}">
LoomVideo: Unifying Multimodal Inputs into Video Generation and Editing
Authors: ,
,
,
,
,
,
,
,
,
,
,
,
,
Abstract
LoomVideo presents an efficient 5B-parameter unified architecture for video generation and editing that reduces computational overhead through novel conditioning mechanisms and multi-modal alignment techniques.
Developing unified video generation and editing models capable of interpreting interleaved multimodal inputs is a promising yet challenging frontier field. Existing unified frameworks predominantly rely on massive models (typically 13B parameters or more) and incorporate source video conditions for editing by concatenating sequence tokens. This concatenation inevitably doubles the sequence length, quadrupling the computational complexity of the self-attention mechanism and introducing prohibitive overhead. To address these bottlenecks, we present LoomVideo, a highly efficient 5B-parameter unified architecture for both video generation and editing. LoomVideo replaces the standard text encoder with a Multimodal Large Language Model (MLLM) and employs Deepstack injection mechanism to align multi-layer MLLM features with the Diffusion Transformer (DiT). Crucially, we introduce a zero-overhead Scale-and-Add conditioning approach for video editing. By scaling and directly adding the clean source video latent to the noised target latent, this elegant design eliminates the need for token concatenation, drastically reducing computational cost while maintaining robust capabilities for complex, non-rigid edits. Furthermore, a Negative Temporal RoPE strategy is seamlessly integrated to handle multiple reference images. Extensive experiments demonstrate that our compact 5B model achieves state-of-the-art or highly competitive performance across comprehensive benchmarks, exhibiting exceptional superiority in e-commerce and fashion generation scenarios. Benefiting from the zero-overhead conditioning mechanism, LoomVideo achieves at least a 5.41x acceleration in inference speed compared to models of similar capabilities, paving the way for highly practical and efficient video foundation models.
Community
Developing unified video generation and editing models capable of interpreting interleaved multimodal inputs is a promising yet challenging frontier field. Existing unified frameworks predominantly rely on massive models (typically 13B parameters or more) and incorporate source video conditions for editing by concatenating sequence tokens. This concatenation inevitably doubles the sequence length, quadrupling the computational complexity of the self-attention mechanism and introducing prohibitive overhead. To address these bottlenecks, we present LoomVideo, a highly efficient 5B-parameter unified architecture for both video generation and editing. LoomVideo replaces the standard text encoder with a Multimodal Large Language Model (MLLM) and employs a Deepstack injection mechanism to align multi-layer MLLM features with the Diffusion Transformer (DiT). Crucially, we introduce a zero-overhead Scale-and-Add conditioning approach for video editing. By scaling and directly adding the clean source video latent to the noised target latent, this elegant design eliminates the need for token concatenation, drastically reducing computational cost while maintaining robust capabilities for complex, non-rigid edits. Furthermore, a Negative Temporal RoPE strategy is seamlessly integrated to handle multiple reference images. Extensive experiments demonstrate that our compact 5B model achieves state-of-the-art or highly competitive performance across comprehensive benchmarks, exhibiting exceptional superiority in e-commerce and fashion generation scenarios. Benefiting from the zero-overhead conditioning mechanism, LoomVideo achieves at least a 5.41× acceleration in inference speed compared to models of similar capabilities, paving the way for highly practical and efficient video foundation models.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2606.06042 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2606.06042 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.