Real-time streaming video-to-video editing (V2V) is critical for interactive applications such as live broadcasting and gaming, yet it remains a formidable challenge due to the stringent requirements for temporal consistency and inference throughput. In this paper, we present SANA-Streaming, a system-algorithm co-designed framework for high-resolution, real-time streaming video editing on consumer GPUs, with the following three core designs: (1) Hybrid Diffusion Transformer architecture introduces softmax attention in part of the blocks to improve local modeling capabilities while preserving the efficiency of linear layers. (2) Cycle-Reverse Regularization is a novel training strategy that enforces semantic consistency by predicting source frames from generated content via flow matching, improving temporal consistency without requiring paired long edited videos. (3) Efficient System Co-design combines fused GDN kernels and Mixed-Precision Quantization (MPQ) optimized for the NVIDIA Blackwell (RTX 5090) architecture. By profiling real-world throughput, our MPQ maximizes Tensor Core utilization while maintaining generation quality. The resulting system achieves real-time 1280 x 704 resolution editing at 24 end-to-end FPS on a single RTX 5090 GPU, with the DiT core running at 58 FPS. Experimental results demonstrate that our co-design approach significantly outperforms existing SOTA methods in both temporal coherence and system throughput.</p>\n","updatedAt":"2026-06-01T02:50:45.919Z","author":{"_id":"64638bd36c27a7e33b26654b","avatarUrl":"/avatars/2ef5aeb94ef7016082975b4cc201873e.svg","fullname":"Yuyang","name":"Yuyang-z","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":6,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8549299836158752},"editors":["Yuyang-z"],"editorAvatarUrls":["/avatars/2ef5aeb94ef7016082975b4cc201873e.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2605.30409","authors":[{"_id":"6a1cf323808ddbc3c7d4349b","name":"Yuyang Zhao","hidden":false},{"_id":"6a1cf323808ddbc3c7d4349c","name":"Yicheng Pan","hidden":false},{"_id":"6a1cf323808ddbc3c7d4349d","name":"Qiyuan He","hidden":false},{"_id":"6a1cf323808ddbc3c7d4349e","name":"Jincheng Yu","hidden":false},{"_id":"6a1cf323808ddbc3c7d4349f","name":"Junsong Chen","hidden":false},{"_id":"6a1cf323808ddbc3c7d434a0","name":"Tian Ye","hidden":false},{"_id":"6a1cf323808ddbc3c7d434a1","name":"Haozhe Liu","hidden":false},{"_id":"6a1cf323808ddbc3c7d434a2","name":"Enze Xie","hidden":false},{"_id":"6a1cf323808ddbc3c7d434a3","name":"Song Han","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/64638bd36c27a7e33b26654b/iBOScL5aQgTdGdwCRHT6A.mp4"],"publishedAt":"2026-05-28T00:00:00.000Z","submittedOnDailyAt":"2026-06-01T00:00:00.000Z","title":"SANA-Streaming: Real-time Streaming Video Editing with Hybrid Diffusion Transformer","submittedOnDailyBy":{"_id":"64638bd36c27a7e33b26654b","avatarUrl":"/avatars/2ef5aeb94ef7016082975b4cc201873e.svg","isPro":false,"fullname":"Yuyang","user":"Yuyang-z","type":"user","name":"Yuyang-z"},"summary":"Real-time streaming video-to-video editing (V2V) is critical for interactive applications such as live broadcasting and gaming, yet it remains a formidable challenge due to the stringent requirements for temporal consistency and inference throughput. In this paper, we present SANA-Streaming, a system-algorithm co-designed framework for high-resolution, real-time streaming video editing on consumer GPUs, with the following three core designs: (1) Hybrid Diffusion Transformer architecture introduces softmax attention in part of the blocks to improve local modeling capabilities while preserving the efficiency of linear layers. (2) Cycle-Reverse Regularization is a novel training strategy that enforces semantic consistency by predicting source frames from generated content via flow matching, improving temporal consistency without requiring paired long edited videos. (3) Efficient System Co-design combines fused GDN kernels and Mixed-Precision Quantization (MPQ) optimized for the NVIDIA Blackwell (RTX 5090) architecture. By profiling real-world throughput, our MPQ maximizes Tensor Core utilization while maintaining generation quality. The resulting system achieves real-time 1280 x 704 resolution editing at 24 end-to-end FPS on a single RTX 5090 GPU, with the DiT core running at 58 FPS. Experimental results demonstrate that our co-design approach significantly outperforms existing SOTA methods in both temporal coherence and system throughput.","upvotes":22,"discussionId":"6a1cf323808ddbc3c7d434a4","projectPage":"https://nvlabs.github.io/Sana/Streaming/","ai_summary":"SANA-Streaming enables real-time high-resolution video-to-video editing through a hybrid diffusion transformer architecture, cycle-reverse regularization, and efficient system co-design optimized for consumer GPUs.","ai_keywords":["diffusion transformer","softmax attention","flow matching","mixed-precision quantization","tensor cores","real-time video editing","temporal consistency","system-algorithm co-design"],"organization":{"_id":"60262b67268c201cdc8b7d43","name":"nvidia","fullname":"NVIDIA","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/65df9200dc3292a8983e5017/Vs5FPVCH-VZBipV3qKTuy.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"64638bd36c27a7e33b26654b","avatarUrl":"/avatars/2ef5aeb94ef7016082975b4cc201873e.svg","isPro":false,"fullname":"Yuyang","user":"Yuyang-z","type":"user"},{"_id":"616961bdd3f656f79ad18ec1","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/616961bdd3f656f79ad18ec1/NYpxas378lDevRuUDepJJ.jpeg","isPro":false,"fullname":"xieenze","user":"xieenze","type":"user"},{"_id":"67470b1dfd46357d9efb7ed6","avatarUrl":"/avatars/1e1e3eed836d9f713208594ffe539e25.svg","isPro":false,"fullname":"Yicheng Pan","user":"RyanPan315464","type":"user"},{"_id":"6a11472b944b9acd42ac85d8","avatarUrl":"/avatars/d092c3e7db19bc8ff0b7616d2603aff1.svg","isPro":false,"fullname":"Mingyu Li","user":"mrvlim","type":"user"},{"_id":"69f4d21845a0ca8f2c077a3c","avatarUrl":"/avatars/325b237fd8c28b9202ca2e5983a3abe9.svg","isPro":false,"fullname":"WideSeek-R1","user":"WideSeek-R1","type":"user"},{"_id":"6039478ab3ecf716b1a5fd4d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg","isPro":true,"fullname":"taesiri","user":"taesiri","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"67136093d2e50f1e8c9fad52","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/0q49MyGuav8lJ9CIeyLhu.png","isPro":false,"fullname":"Donghao Zhou","user":"donghao-zhou","type":"user"},{"_id":"69af90ad624002cfc5363763","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/beczqkm0jdO3on4nC5o8s.png","isPro":true,"fullname":"Alexander Ezharjan","user":"ezharjan","type":"user"},{"_id":"66015e8aa4d296af07de538e","avatarUrl":"/avatars/a1295c631cc2646282c545859975ce4c.svg","isPro":false,"fullname":"Owen","user":"Owen777","type":"user"},{"_id":"62dbeaf3d36b2070f922747f","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1671954059773-62dbeaf3d36b2070f922747f.jpeg","isPro":false,"fullname":"Junyao Hu","user":"hujunyao","type":"user"},{"_id":"632c2100ea6e62428ab201e9","avatarUrl":"/avatars/5bd355e095af93261928198e3d6d5696.svg","isPro":false,"fullname":"Xingzhaohu","user":"xingzhaohu","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"60262b67268c201cdc8b7d43","name":"nvidia","fullname":"NVIDIA","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/65df9200dc3292a8983e5017/Vs5FPVCH-VZBipV3qKTuy.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2605/2605.30409.md"}">
SANA-Streaming: Real-time Streaming Video Editing with Hybrid Diffusion Transformer
Published on May 28
· Submitted by Yuyang on Jun 1 Abstract
SANA-Streaming enables real-time high-resolution video-to-video editing through a hybrid diffusion transformer architecture, cycle-reverse regularization, and efficient system co-design optimized for consumer GPUs.
AI-generated summary
Real-time streaming video-to-video editing (V2V) is critical for interactive applications such as live broadcasting and gaming, yet it remains a formidable challenge due to the stringent requirements for temporal consistency and inference throughput. In this paper, we present SANA-Streaming, a system-algorithm co-designed framework for high-resolution, real-time streaming video editing on consumer GPUs, with the following three core designs: (1) Hybrid Diffusion Transformer architecture introduces softmax attention in part of the blocks to improve local modeling capabilities while preserving the efficiency of linear layers. (2) Cycle-Reverse Regularization is a novel training strategy that enforces semantic consistency by predicting source frames from generated content via flow matching, improving temporal consistency without requiring paired long edited videos. (3) Efficient System Co-design combines fused GDN kernels and Mixed-Precision Quantization (MPQ) optimized for the NVIDIA Blackwell (RTX 5090) architecture. By profiling real-world throughput, our MPQ maximizes Tensor Core utilization while maintaining generation quality. The resulting system achieves real-time 1280 x 704 resolution editing at 24 end-to-end FPS on a single RTX 5090 GPU, with the DiT core running at 58 FPS. Experimental results demonstrate that our co-design approach significantly outperforms existing SOTA methods in both temporal coherence and system throughput.
Community
Real-time streaming video-to-video editing (V2V) is critical for interactive applications such as live broadcasting and gaming, yet it remains a formidable challenge due to the stringent requirements for temporal consistency and inference throughput. In this paper, we present SANA-Streaming, a system-algorithm co-designed framework for high-resolution, real-time streaming video editing on consumer GPUs, with the following three core designs: (1) Hybrid Diffusion Transformer architecture introduces softmax attention in part of the blocks to improve local modeling capabilities while preserving the efficiency of linear layers. (2) Cycle-Reverse Regularization is a novel training strategy that enforces semantic consistency by predicting source frames from generated content via flow matching, improving temporal consistency without requiring paired long edited videos. (3) Efficient System Co-design combines fused GDN kernels and Mixed-Precision Quantization (MPQ) optimized for the NVIDIA Blackwell (RTX 5090) architecture. By profiling real-world throughput, our MPQ maximizes Tensor Core utilization while maintaining generation quality. The resulting system achieves real-time 1280 x 704 resolution editing at 24 end-to-end FPS on a single RTX 5090 GPU, with the DiT core running at 58 FPS. Experimental results demonstrate that our co-design approach significantly outperforms existing SOTA methods in both temporal coherence and system throughput.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2605.30409 in a model README.md to link it from this page.
Cite arxiv.org/abs/2605.30409 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2605.30409 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.