We introduce SANA-Video 2.0, a hybrid video diffusion transformer instantiated at 5B and 14B scales under a unified architecture. Designed to generate high-quality video up to 720p on a single GPU, SANA-Video 2.0 matches full-softmax video DiTs in quality while retaining the favorable long-sequence scaling of linear attention. To avoid quadratic attention throughout, Hybrid Linear-Softmax Attention combines gated linear attention for O(N)-dominated mixing with periodic gated-softmax anchors at a 3:1 ratio, restoring the full-rank token interactions that pure linear attention lacks. To propagate these refreshed representations across depth, Block Attention Residuals (AttnRes) route completed block summaries into later linear layers, enabling anchor-feature reuse and boosting deep-layer effective rank by ~12%. Through from-scratch training, SANA-Video 2.0 learns the complete hybrid directly rather than linearizing pretrained models, with reduced-resolution proxy studies establishing 25% softmax as the optimal quality-efficiency trade-off. With 40-step sampling, SANA-Video 2.0 achieves a VBench score of 84.30 in 13.2s at 480p on a single H100, remaining competitive with far larger softmax video DiTs at a fraction of the latency. Its compiled DiT forward pass is 3.2x faster than a matched full-softmax baseline at 720p/60s, a gap that expands with video duration. Furthermore, full-stack Sol-Engine optimization (kernel fusion, caching, and sparse attention) accelerates this hardware-friendly backbone by a further 3.58x, bringing the 5B pipeline to 13.06s at 720p/5s and making it 120x faster than Wan 2.2-A14B on one H100. Overall, our hybrid design recovers softmax-level expressiveness at substantially reduced cost, unlocking scalable long, high resolution video generation.<br><a href=\"https://cdn-uploads.huggingface.co/production/uploads/645b5b09bc7518912e1f9733/glRRIsaaPlqTEInLR0Zwm.png\" rel=\"nofollow\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/645b5b09bc7518912e1f9733/glRRIsaaPlqTEInLR0Zwm.png\" alt=\"image\"></a></p>\n","updatedAt":"2026-07-24T11:33:30.604Z","author":{"_id":"645b5b09bc7518912e1f9733","avatarUrl":"/avatars/4d35f728b41f93881a9b67c337f4d1df.svg","fullname":"Chen","name":"Lawrence-cj","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":17,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8435601592063904},"editors":["Lawrence-cj"],"editorAvatarUrls":["/avatars/4d35f728b41f93881a9b67c337f4d1df.svg"],"reactions":[],"isReport":false}},{"id":"6a634d994b6e26cc9eff2710","author":{"_id":"645b5b09bc7518912e1f9733","avatarUrl":"/avatars/4d35f728b41f93881a9b67c337f4d1df.svg","fullname":"Chen","name":"Lawrence-cj","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":17,"isUserFollowing":false},"createdAt":"2026-07-24T11:33:45.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"GitHub: https://github.com/NVlabs/Sana/","html":"<p>GitHub: <a href=\"https://github.com/NVlabs/Sana/\" rel=\"nofollow\">https://github.com/NVlabs/Sana/</a></p>\n","updatedAt":"2026-07-24T11:33:45.409Z","author":{"_id":"645b5b09bc7518912e1f9733","avatarUrl":"/avatars/4d35f728b41f93881a9b67c337f4d1df.svg","fullname":"Chen","name":"Lawrence-cj","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":17,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.6664382219314575},"editors":["Lawrence-cj"],"editorAvatarUrls":["/avatars/4d35f728b41f93881a9b67c337f4d1df.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2607.21553","authors":[{"_id":"6a632d573f6711f3a0ba9359","name":"Junsong Chen","hidden":false},{"_id":"6a632d573f6711f3a0ba935a","name":"Jincheng Yu","hidden":false},{"_id":"6a632d573f6711f3a0ba935b","name":"Yitong Li","hidden":false},{"_id":"6a632d573f6711f3a0ba935c","name":"Shuchen Xue","hidden":false},{"_id":"6a632d573f6711f3a0ba935d","name":"Haozhe Liu","hidden":false},{"_id":"6a632d573f6711f3a0ba935e","name":"Jingyu Xin","hidden":false},{"_id":"6a632d573f6711f3a0ba935f","name":"Yuyang Zhao","hidden":false},{"_id":"6a632d573f6711f3a0ba9360","name":"Tian Ye","hidden":false},{"_id":"6a632d573f6711f3a0ba9361","name":"Zhangjie Wu","hidden":false},{"_id":"6a632d573f6711f3a0ba9362","name":"Zian Wang","hidden":false},{"_id":"6a632d573f6711f3a0ba9363","name":"Daquan Zhou","hidden":false},{"_id":"6a632d573f6711f3a0ba9364","name":"Ping Luo","hidden":false},{"_id":"6a632d573f6711f3a0ba9365","name":"Song Han","hidden":false},{"_id":"6a632d573f6711f3a0ba9366","name":"Enze Xie","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/645b5b09bc7518912e1f9733/8WUF7Xm9QNEHHKgPGrfNO.mp4"],"publishedAt":"2026-07-23T00:00:00.000Z","submittedOnDailyAt":"2026-07-24T00:00:00.000Z","title":"SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation","submittedOnDailyBy":{"_id":"645b5b09bc7518912e1f9733","avatarUrl":"/avatars/4d35f728b41f93881a9b67c337f4d1df.svg","isPro":false,"fullname":"Chen","user":"Lawrence-cj","type":"user","name":"Lawrence-cj"},"summary":"We introduce SANA-Video 2.0, a hybrid video diffusion transformer instantiated at 5B and 14B scales under a unified architecture. Designed to generate high-quality video up to 720p on a single GPU, SANA-Video 2.0 matches full-softmax video DiTs in quality while retaining the favorable long-sequence scaling of linear attention. To avoid quadratic attention throughout, Hybrid Linear-Softmax Attention combines gated linear attention for O(N)-dominated mixing with periodic gated-softmax anchors at a 3:1 ratio, restoring the full-rank token interactions that pure linear attention lacks. To propagate these refreshed representations across depth, Block Attention Residuals (AttnRes) route completed block summaries into later linear layers, enabling anchor-feature reuse and boosting deep-layer effective rank by ~12%. Through from-scratch training, SANA-Video 2.0 learns the complete hybrid directly rather than linearizing pretrained models, with reduced-resolution proxy studies establishing 25% softmax as the optimal quality-efficiency trade-off. With 40-step sampling, SANA-Video 2.0 achieves a VBench score of 84.30 in 13.2s at 480p on a single H100, remaining competitive with far larger softmax video DiTs at a fraction of the latency. Its compiled DiT forward pass is 3.2x faster than a matched full-softmax baseline at 720p/60s, a gap that expands with video duration. Furthermore, full-stack Sol-Engine optimization (kernel fusion, caching, and sparse attention) accelerates this hardware-friendly backbone by a further 3.58x, bringing the 5B pipeline to 13.06s at 720p/5s and making it 120x faster than Wan 2.2-A14B on one H100. Overall, our hybrid design recovers softmax-level expressiveness at substantially reduced cost, unlocking scalable long, high resolution video generation.","upvotes":13,"discussionId":"6a632d583f6711f3a0ba9367","projectPage":"https://nvlabs.github.io/Sana/Video2","organization":{"_id":"60262b67268c201cdc8b7d43","name":"nvidia","fullname":"NVIDIA","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/65df9200dc3292a8983e5017/Vs5FPVCH-VZBipV3qKTuy.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"603bdba23249b99991dbcbc4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/603bdba23249b99991dbcbc4/cxCnN1H-RXOhojHY3Wcxo.jpeg","isPro":false,"fullname":"Tolga Cangöz","user":"tolgacangoz","type":"user"},{"_id":"69830d947f99218c71ed40d1","avatarUrl":"/avatars/2808859b43756bc29281336f5e60c1ba.svg","isPro":false,"fullname":"Claire Duval","user":"abc-456","type":"user"},{"_id":"645b5b09bc7518912e1f9733","avatarUrl":"/avatars/4d35f728b41f93881a9b67c337f4d1df.svg","isPro":false,"fullname":"Chen","user":"Lawrence-cj","type":"user"},{"_id":"66015e8aa4d296af07de538e","avatarUrl":"/avatars/a1295c631cc2646282c545859975ce4c.svg","isPro":false,"fullname":"Owen","user":"Owen777","type":"user"},{"_id":"6641dba91a96431427a006ce","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6641dba91a96431427a006ce/uKOGbZaxi0AYNJpCznLsn.jpeg","isPro":false,"fullname":"Zhiqiu Zhang","user":"ZZQ987","type":"user"},{"_id":"661f9a1b142a51d630dc165d","avatarUrl":"/avatars/b0d5325c4ce7cb632bc22766ebbecdec.svg","isPro":false,"fullname":"whalelin_Lin","user":"whalelin","type":"user"},{"_id":"64d5c6acdd57652c1a472f2d","avatarUrl":"/avatars/358ea808645b5bf72dd82b07cacf7a78.svg","isPro":false,"fullname":"Xiong Xuyuan","user":"xjxyys","type":"user"},{"_id":"68bb0e42a14418015d344509","avatarUrl":"/avatars/35982535035b4ad618c88a7c89fe1d37.svg","isPro":true,"fullname":"NAMENAME","user":"VLAD545645645","type":"user"},{"_id":"62919485a29097b211bc7b83","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/62919485a29097b211bc7b83/TX8iBGu5JSuFlrRvjPEBV.png","isPro":false,"fullname":"YukangChen","user":"Yukang","type":"user"},{"_id":"6a5a369a80d584889c3d5d3d","avatarUrl":"/avatars/eb0f7e4f64e366b3fb8a83fc9a620102.svg","isPro":false,"fullname":"Vladimir","user":"vstiff","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"5f17f0a0925b9863e28ad517","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/5f17f0a0925b9863e28ad517/78BR8GuQiX9a8wbbRlSAC.webp","isPro":true,"fullname":"Victor Mustar","user":"victor","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"60262b67268c201cdc8b7d43","name":"nvidia","fullname":"NVIDIA","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/65df9200dc3292a8983e5017/Vs5FPVCH-VZBipV3qKTuy.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2607/2607.21553.md","query":{}}">
SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation
Published on Jul 23
· Submitted by Chen on Jul 24 Abstract
We introduce SANA-Video 2.0, a hybrid video diffusion transformer instantiated at 5B and 14B scales under a unified architecture. Designed to generate high-quality video up to 720p on a single GPU, SANA-Video 2.0 matches full-softmax video DiTs in quality while retaining the favorable long-sequence scaling of linear attention. To avoid quadratic attention throughout, Hybrid Linear-Softmax Attention combines gated linear attention for O(N)-dominated mixing with periodic gated-softmax anchors at a 3:1 ratio, restoring the full-rank token interactions that pure linear attention lacks. To propagate these refreshed representations across depth, Block Attention Residuals (AttnRes) route completed block summaries into later linear layers, enabling anchor-feature reuse and boosting deep-layer effective rank by ~12%. Through from-scratch training, SANA-Video 2.0 learns the complete hybrid directly rather than linearizing pretrained models, with reduced-resolution proxy studies establishing 25% softmax as the optimal quality-efficiency trade-off. With 40-step sampling, SANA-Video 2.0 achieves a VBench score of 84.30 in 13.2s at 480p on a single H100, remaining competitive with far larger softmax video DiTs at a fraction of the latency. Its compiled DiT forward pass is 3.2x faster than a matched full-softmax baseline at 720p/60s, a gap that expands with video duration. Furthermore, full-stack Sol-Engine optimization (kernel fusion, caching, and sparse attention) accelerates this hardware-friendly backbone by a further 3.58x, bringing the 5B pipeline to 13.06s at 720p/5s and making it 120x faster than Wan 2.2-A14B on one H100. Overall, our hybrid design recovers softmax-level expressiveness at substantially reduced cost, unlocking scalable long, high resolution video generation.
Community
We introduce SANA-Video 2.0, a hybrid video diffusion transformer instantiated at 5B and 14B scales under a unified architecture. Designed to generate high-quality video up to 720p on a single GPU, SANA-Video 2.0 matches full-softmax video DiTs in quality while retaining the favorable long-sequence scaling of linear attention. To avoid quadratic attention throughout, Hybrid Linear-Softmax Attention combines gated linear attention for O(N)-dominated mixing with periodic gated-softmax anchors at a 3:1 ratio, restoring the full-rank token interactions that pure linear attention lacks. To propagate these refreshed representations across depth, Block Attention Residuals (AttnRes) route completed block summaries into later linear layers, enabling anchor-feature reuse and boosting deep-layer effective rank by ~12%. Through from-scratch training, SANA-Video 2.0 learns the complete hybrid directly rather than linearizing pretrained models, with reduced-resolution proxy studies establishing 25% softmax as the optimal quality-efficiency trade-off. With 40-step sampling, SANA-Video 2.0 achieves a VBench score of 84.30 in 13.2s at 480p on a single H100, remaining competitive with far larger softmax video DiTs at a fraction of the latency. Its compiled DiT forward pass is 3.2x faster than a matched full-softmax baseline at 720p/60s, a gap that expands with video duration. Furthermore, full-stack Sol-Engine optimization (kernel fusion, caching, and sparse attention) accelerates this hardware-friendly backbone by a further 3.58x, bringing the 5B pipeline to 13.06s at 720p/5s and making it 120x faster than Wan 2.2-A14B on one H100. Overall, our hybrid design recovers softmax-level expressiveness at substantially reduced cost, unlocking scalable long, high resolution video generation.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2607.21553 in a model README.md to link it from this page.
Cite arxiv.org/abs/2607.21553 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2607.21553 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.