On-policy distillation, in which a teacher corrects samples that the student itself generates, presupposes that the two models speak the same language: identical VAE latents, matching architectures, and a common timestep grid. We ask what happens when none of this holds, as when the strongest teacher available and the student one wishes to deploy come from different model families, and find that the standard recipes have no answer: teacher latents cannot serve as targets in a foreign coordinate system, per-pixel losses against a teacher that stochastically re-draws local detail degenerate into blur or divergence, and timestep indices lose their meaning across mismatched schedules. We present Any-OPD, to our knowledge the first framework for on-policy distillation between arbitrary pairs of latent flow-matching generators. Any-OPD treats the teacher purely as a black-box sampler and connects the two models at exactly one point: a frozen, model-agnostic vision representation in which their independently decoded outputs are compared, sidestepping every assumption about latents, features, or architecture. Trajectory correspondence is recovered by matching continuous noise levels instead of step indices, and a brief anchoring phase, in which teacher samples are re-encoded through the student's own VAE, ensures the on-policy gradient measures sample quality rather than domain mismatch. Distilling the 12B FLUX.1-dev into the 2.5B SD3.5-Medium, Any-OPD lifts the student's PickScore from 0.846 to 0.884 and HPSv3 from 9.12 to 10.97, rivaling the teacher at a fifth of its size, where direct latent regression fails to train at all.</p>\n","updatedAt":"2026-08-05T01:54:46.469Z","author":{"_id":"6485dd6d07a2c1915060f603","avatarUrl":"/avatars/8594d647359a7d19ab29b8ec91d1444e.svg","fullname":"fu","name":"simingfu","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9371470212936401},"editors":["simingfu"],"editorAvatarUrls":["/avatars/8594d647359a7d19ab29b8ec91d1444e.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.03316","authors":[{"_id":"6a7297781a375f948521c340","name":"Siming Fu","hidden":false},{"_id":"6a7297781a375f948521c341","name":"Zheming Fu","hidden":false},{"_id":"6a7297781a375f948521c342","name":"Ruizhe He","hidden":false},{"_id":"6a7297781a375f948521c343","name":"Hualiang Wang","hidden":false},{"_id":"6a7297781a375f948521c344","name":"Jie Huang","hidden":false},{"_id":"6a7297781a375f948521c345","name":"Xiaoxiao Ma","hidden":false},{"_id":"6a7297781a375f948521c346","name":"Mingchen Zhong","hidden":false},{"_id":"6a7297781a375f948521c347","name":"Weihu Huang","hidden":false},{"_id":"6a7297781a375f948521c348","name":"Xiaoxuan He","hidden":false},{"_id":"6a7297781a375f948521c349","name":"Haojun Xu","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/6485dd6d07a2c1915060f603/7XawnnUdtoDbWyxP7FhYm.png"],"publishedAt":"2026-08-04T00:00:00.000Z","submittedOnDailyAt":"2026-08-05T00:00:00.000Z","title":"Any-OPD: Heterogeneous On-Policy Distillation for Flow-Matching Models via Representation-Space Bridging","submittedOnDailyBy":{"_id":"6485dd6d07a2c1915060f603","avatarUrl":"/avatars/8594d647359a7d19ab29b8ec91d1444e.svg","isPro":false,"fullname":"fu","user":"simingfu","type":"user","name":"simingfu"},"summary":"On-policy distillation, in which a teacher corrects samples that the student itself generates, presupposes that the two models speak the same language: identical VAE latents, matching architectures, and a common timestep grid. We ask what happens when none of this holds, as when the strongest teacher available and the student one wishes to deploy come from different model families, and find that the standard recipes have no answer: teacher latents cannot serve as targets in a foreign coordinate system, per-pixel losses against a teacher that stochastically re-draws local detail degenerate into blur or divergence, and timestep indices lose their meaning across mismatched schedules. We present Any-OPD, to our knowledge the first framework for on-policy distillation between arbitrary pairs of latent flow-matching generators. Any-OPD treats the teacher purely as a black-box sampler and connects the two models at exactly one point: a frozen, model-agnostic vision representation in which their independently decoded outputs are compared, sidestepping every assumption about latents, features, or architecture. Trajectory correspondence is recovered by matching continuous noise levels instead of step indices, and a brief anchoring phase, in which teacher samples are re-encoded through the student's own VAE, ensures the on-policy gradient measures sample quality rather than domain mismatch. Distilling the 12B FLUX.1-dev into the 2.5B SD3.5-Medium, Any-OPD lifts the student's PickScore from 0.846 to 0.884 and HPSv3 from 9.12 to 10.97, rivaling the teacher at a fifth of its size, where direct latent regression fails to train at all.","upvotes":13,"discussionId":"6a7297781a375f948521c34a","organization":{"_id":"682a97a154b087448a5504ee","name":"jingdong1","fullname":"jingdong","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/682a966182dc4fc3e373e3ed/9xUgXZCij6qGJpLqpR99M.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6485dd6d07a2c1915060f603","avatarUrl":"/avatars/8594d647359a7d19ab29b8ec91d1444e.svg","isPro":false,"fullname":"fu","user":"simingfu","type":"user"},{"_id":"6039478ab3ecf716b1a5fd4d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg","isPro":true,"fullname":"taesiri","user":"taesiri","type":"user"},{"_id":"66974e7a57a5a55a1f4a0469","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/66974e7a57a5a55a1f4a0469/76WAtxoxJO5rjZWxEmF2t.jpeg","isPro":false,"fullname":"xxh","user":"shreddedpork","type":"user"},{"_id":"6a6a831d726441725a2ed822","avatarUrl":"/avatars/45ba67b0e43f5f2e65aff93418dcdba2.svg","isPro":false,"fullname":"Charles Thompson","user":"charlesthompson","type":"user"},{"_id":"6a6a9417a569279e390e1343","avatarUrl":"/avatars/4866974d3fab4ba081b094cd0819de75.svg","isPro":false,"fullname":"Mark Davis","user":"silverArcL","type":"user"},{"_id":"6a6c7a3d3574d63d5305cd74","avatarUrl":"/avatars/78a0b7bd8c1b9996c19966b58df44488.svg","isPro":false,"fullname":"Richard Wilson","user":"Cobalt-Richard4","type":"user"},{"_id":"6a6c84f892fd458eaa9c7938","avatarUrl":"/avatars/6390b9321a236bf70ff45cf3b85a0ccf.svg","isPro":false,"fullname":"Jennifer Williams","user":"velvetscope","type":"user"},{"_id":"6a6d434fe4c3f57705bd16d3","avatarUrl":"/avatars/8b44133480902074481bc126ea367b75.svg","isPro":false,"fullname":"Mary Thompson","user":"mary-thompson","type":"user"},{"_id":"6a6aa77d342f6961a3ac5feb","avatarUrl":"/avatars/5b391a7dd212e505964da69ce2f00d24.svg","isPro":false,"fullname":"David Lopez","user":"richard-7039428","type":"user"},{"_id":"6a6de9f3b34a8844196674a7","avatarUrl":"/avatars/92a2fc49097826280da72bdbbcc1112d.svg","isPro":false,"fullname":"Paul Sanchez","user":"sam-7519356","type":"user"},{"_id":"6a701ed9418ef8ed5ce8852b","avatarUrl":"/avatars/2bfcc94945d474ed6e2bf99e9e8f8095.svg","isPro":false,"fullname":"Thomas White","user":"david-0396088","type":"user"},{"_id":"63ea05c020c1097187153b57","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1676281239163-noauth.jpeg","isPro":false,"fullname":"Qin Zhou","user":"Matrix53","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"682a97a154b087448a5504ee","name":"jingdong1","fullname":"jingdong","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/682a966182dc4fc3e373e3ed/9xUgXZCij6qGJpLqpR99M.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.03316.md","query":{}}">
Any-OPD: Heterogeneous On-Policy Distillation for Flow-Matching Models via Representation-Space Bridging
Published on Aug 4
· Submitted by fu on Aug 5 Abstract
On-policy distillation, in which a teacher corrects samples that the student itself generates, presupposes that the two models speak the same language: identical VAE latents, matching architectures, and a common timestep grid. We ask what happens when none of this holds, as when the strongest teacher available and the student one wishes to deploy come from different model families, and find that the standard recipes have no answer: teacher latents cannot serve as targets in a foreign coordinate system, per-pixel losses against a teacher that stochastically re-draws local detail degenerate into blur or divergence, and timestep indices lose their meaning across mismatched schedules. We present Any-OPD, to our knowledge the first framework for on-policy distillation between arbitrary pairs of latent flow-matching generators. Any-OPD treats the teacher purely as a black-box sampler and connects the two models at exactly one point: a frozen, model-agnostic vision representation in which their independently decoded outputs are compared, sidestepping every assumption about latents, features, or architecture. Trajectory correspondence is recovered by matching continuous noise levels instead of step indices, and a brief anchoring phase, in which teacher samples are re-encoded through the student's own VAE, ensures the on-policy gradient measures sample quality rather than domain mismatch. Distilling the 12B FLUX.1-dev into the 2.5B SD3.5-Medium, Any-OPD lifts the student's PickScore from 0.846 to 0.884 and HPSv3 from 9.12 to 10.97, rivaling the teacher at a fifth of its size, where direct latent regression fails to train at all.
Community
On-policy distillation, in which a teacher corrects samples that the student itself generates, presupposes that the two models speak the same language: identical VAE latents, matching architectures, and a common timestep grid. We ask what happens when none of this holds, as when the strongest teacher available and the student one wishes to deploy come from different model families, and find that the standard recipes have no answer: teacher latents cannot serve as targets in a foreign coordinate system, per-pixel losses against a teacher that stochastically re-draws local detail degenerate into blur or divergence, and timestep indices lose their meaning across mismatched schedules. We present Any-OPD, to our knowledge the first framework for on-policy distillation between arbitrary pairs of latent flow-matching generators. Any-OPD treats the teacher purely as a black-box sampler and connects the two models at exactly one point: a frozen, model-agnostic vision representation in which their independently decoded outputs are compared, sidestepping every assumption about latents, features, or architecture. Trajectory correspondence is recovered by matching continuous noise levels instead of step indices, and a brief anchoring phase, in which teacher samples are re-encoded through the student's own VAE, ensures the on-policy gradient measures sample quality rather than domain mismatch. Distilling the 12B FLUX.1-dev into the 2.5B SD3.5-Medium, Any-OPD lifts the student's PickScore from 0.846 to 0.884 and HPSv3 from 9.12 to 10.97, rivaling the teacher at a fifth of its size, where direct latent regression fails to train at all.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2608.03316 in a model README.md to link it from this page.
Cite arxiv.org/abs/2608.03316 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2608.03316 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.