The underlying mechanism behind the evolution from SRA to Self-Flow</p>\n","updatedAt":"2026-07-03T03:52:01.925Z","author":{"_id":"662a0f2d4bab737c1a279843","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/662a0f2d4bab737c1a279843/fC2p3mjMHkVpDQdEqkuR4.png","fullname":"Dengyang Jiang","name":"DyJiang","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":16,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9114007353782654},"editors":["DyJiang"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/662a0f2d4bab737c1a279843/fC2p3mjMHkVpDQdEqkuR4.png"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2607.02508","authors":[{"_id":"6a4731866ee372f6920de356","name":"Dengyang Jiang","hidden":false},{"_id":"6a4731866ee372f6920de357","name":"Mengmeng Wang","hidden":false},{"_id":"6a4731866ee372f6920de358","name":"Harry Yang","hidden":false},{"_id":"6a4731866ee372f6920de359","name":"Jingdong Wang","hidden":false}],"publishedAt":"2026-07-02T00:00:00.000Z","submittedOnDailyAt":"2026-07-03T00:00:00.000Z","title":"From SRA to Self-Flow: Data Augmentation or Self-Supervision?","submittedOnDailyBy":{"_id":"662a0f2d4bab737c1a279843","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/662a0f2d4bab737c1a279843/fC2p3mjMHkVpDQdEqkuR4.png","isPro":false,"fullname":"Dengyang Jiang","user":"DyJiang","type":"user","name":"DyJiang"},"summary":"Representation alignment has become an effective way to accelerate diffusion transformer training and improve generation quality. Recent self-alignment methods, such as SRA and Self-Flow, further remove the dependency on external pretrained encoders by constructing alignment within the diffusion model itself. However, the mechanism behind the improvement from SRA to Self-Flow, dual-time scheduling, remains under-examined: Self-Flow attributes its gain to interactions between tokens at different noise levels, where cleaner tokens help infer noisier ones. In this work, we revisit this explanation and ask whether the gain instead comes from data augmentation along the noise dimension. To disentangle these factors, we introduce Attention Separation, which preserves the same dual-timestep input as Self-Flow while blocking attention between tokens assigned to different noise levels. Surprisingly, removing such interaction does not degrade performance and can even improve it, suggesting that the improvement from SRA to Self-Flow mainly comes from data augmentation. Furthermore,We show that Attention Separation itself provides an augmentation effect by splitting a single image into multiple effective training parts to expand the training data. Based on these observations, we combine self-representation alignment with dual-timestep and attention-separation augmentation, and demonstrate the effectiveness of this design on ImageNet.","upvotes":8,"discussionId":"6a4731866ee372f6920de35a","ai_summary":"Research investigates the mechanisms behind self-alignment methods in diffusion transformers, finding that performance improvements stem primarily from data augmentation along the noise dimension rather than token interactions between noise levels.","ai_keywords":["representation alignment","diffusion transformer","self-alignment","SRA","Self-Flow","dual-time scheduling","attention separation","data augmentation","noise dimension","token interactions"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct"},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"662a0f2d4bab737c1a279843","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/662a0f2d4bab737c1a279843/fC2p3mjMHkVpDQdEqkuR4.png","isPro":false,"fullname":"Dengyang Jiang","user":"DyJiang","type":"user"},{"_id":"64a54586c0f13de8e7093314","avatarUrl":"/avatars/389e43e9a32cf2fc95f8f3a23b8f0508.svg","isPro":false,"fullname":"Ruoyi Du","user":"RuoyiDu","type":"user"},{"_id":"65f2e3d1cec22d29ce41ef94","avatarUrl":"/avatars/2ab7091ad7eb45ef8d813f63c1d20713.svg","isPro":false,"fullname":"xmz","user":"xmz111","type":"user"},{"_id":"6a2da6c8ca070ee12c6e396c","avatarUrl":"/avatars/0355287dcabaa67dbc7f0b10b87451f9.svg","isPro":false,"fullname":"Joe Mama","user":"JoeMama123123123","type":"user"},{"_id":"66615c855fd9d736e670e0a9","avatarUrl":"/avatars/0ff3127b513552432a7c651e21d7f283.svg","isPro":false,"fullname":"wangshuai","user":"wangsssssss","type":"user"},{"_id":"646f1bef075e11ca78da3bb7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/646f1bef075e11ca78da3bb7/gNS-ikyZXYeMrf4a7HTQE.jpeg","isPro":false,"fullname":"Dongyang Liu (Chris Liu)","user":"Cxxs","type":"user"},{"_id":"6486df66373f79a52913e017","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6486df66373f79a52913e017/vUncXohxJN4ixXR6QUMxh.jpeg","isPro":true,"fullname":"Xiangpeng Yang","user":"XiangpengYang","type":"user"},{"_id":"699eda17233213e661d69edb","avatarUrl":"/avatars/72e15701bad7d6b9ba6a62837ea15624.svg","isPro":false,"fullname":"韩千怡","user":"noahga","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2607/2607.02508.md","query":{}}">
From SRA to Self-Flow: Data Augmentation or Self-Supervision?
Abstract
Research investigates the mechanisms behind self-alignment methods in diffusion transformers, finding that performance improvements stem primarily from data augmentation along the noise dimension rather than token interactions between noise levels.
Representation alignment has become an effective way to accelerate diffusion transformer training and improve generation quality. Recent self-alignment methods, such as SRA and Self-Flow, further remove the dependency on external pretrained encoders by constructing alignment within the diffusion model itself. However, the mechanism behind the improvement from SRA to Self-Flow, dual-time scheduling, remains under-examined: Self-Flow attributes its gain to interactions between tokens at different noise levels, where cleaner tokens help infer noisier ones. In this work, we revisit this explanation and ask whether the gain instead comes from data augmentation along the noise dimension. To disentangle these factors, we introduce Attention Separation, which preserves the same dual-timestep input as Self-Flow while blocking attention between tokens assigned to different noise levels. Surprisingly, removing such interaction does not degrade performance and can even improve it, suggesting that the improvement from SRA to Self-Flow mainly comes from data augmentation. Furthermore,We show that Attention Separation itself provides an augmentation effect by splitting a single image into multiple effective training parts to expand the training data. Based on these observations, we combine self-representation alignment with dual-timestep and attention-separation augmentation, and demonstrate the effectiveness of this design on ImageNet.
Community
The underlying mechanism behind the evolution from SRA to Self-Flow
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2607.02508 in a model README.md to link it from this page.
Cite arxiv.org/abs/2607.02508 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2607.02508 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.