DAPD aligns on-policy and reference behavior under matched information conditions.</p>\n","updatedAt":"2026-08-04T05:59:11.113Z","author":{"_id":"65cd955637be1841d0b75397","avatarUrl":"/avatars/9f39725cb42d9276899fd58a91468d6a.svg","fullname":"Jianyu Wu","name":"Uanu","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7871206998825073},"editors":["Uanu"],"editorAvatarUrls":["/avatars/9f39725cb42d9276899fd58a91468d6a.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.01735","authors":[{"_id":"6a7179f1ec5082b9f872cf16","name":"Jianyu Wu","hidden":false},{"_id":"6a7179f1ec5082b9f872cf17","name":"Yizhou Wang","hidden":false},{"_id":"6a7179f1ec5082b9f872cf18","name":"Encheng Su","hidden":false},{"_id":"6a7179f1ec5082b9f872cf19","name":"Chen Tang","hidden":false},{"_id":"6a7179f1ec5082b9f872cf1a","name":"Shixiang Tang","hidden":false}],"publishedAt":"2026-08-03T00:00:00.000Z","submittedOnDailyAt":"2026-08-04T00:00:00.000Z","title":"DAPD: Dual-Anchored Policy Distillation","submittedOnDailyBy":{"_id":"65cd955637be1841d0b75397","avatarUrl":"/avatars/9f39725cb42d9276899fd58a91468d6a.svg","isPro":false,"fullname":"Jianyu Wu","user":"Uanu","type":"user","name":"Uanu"},"summary":"On-policy (self) distillation (OPSD) is increasingly adopted for language-model post-training. It strengthens the teacher with privileged information but can induce a privilege illusion: the student learns privilege-dependent behavior it cannot reproduce from its inference-time context, yet behaves as if the training-time privileged information remained available, ultimately degrading performance. In this paper, we identify information asymmetry between the privileged teacher and the student at inference as the root cause of this failure in OPSD. To resolve this asymmetry, we propose Dual-Anchored Policy Distillation (DAPD), a unified framework with two levels of anchoring. Dual-Path Anchoring (DPA) introduces a self-conditioned bridge and aligns reference and rollout behavior along two matched-information paths, preventing privilege-dependent behavior from being transferred to the inference-time student. Dual-Source Anchoring (DSA) applies these paths in both reference-to-rollout and rollout-to-reference directions, reducing reliance on privileged reference guidance while preserving correctness supervision. Extensive experiments show that DAPD significantly alleviates privilege illusion, outperforming OPSD on Qwen3-4B by +2.00 points on average across tasks. Notably, its gains persist across scales, reaching +2.69 at 4B and +2.78 at 32B.","upvotes":2,"discussionId":"6a7179f1ec5082b9f872cf1b","organization":{"_id":"6a4fb75a1c66dbf208e7ddb6","name":"Shanghai-AI-Laboratory","fullname":"Shanghai AI Laboratory","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/65cd955637be1841d0b75397/Rao_Kq6NMtTVfSqLUIR4k.webp"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"65cd955637be1841d0b75397","avatarUrl":"/avatars/9f39725cb42d9276899fd58a91468d6a.svg","isPro":false,"fullname":"Jianyu Wu","user":"Uanu","type":"user"},{"_id":"664d930f4b870dd167473c1c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/664d930f4b870dd167473c1c/TXVEPGvkhftdI_xE1mluu.jpeg","isPro":false,"fullname":"Andy Guan","user":"andytonglove","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"6a4fb75a1c66dbf208e7ddb6","name":"Shanghai-AI-Laboratory","fullname":"Shanghai AI Laboratory","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/65cd955637be1841d0b75397/Rao_Kq6NMtTVfSqLUIR4k.webp"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.01735.md","query":{}}">
DAPD: Dual-Anchored Policy Distillation
Abstract
On-policy (self) distillation (OPSD) is increasingly adopted for language-model post-training. It strengthens the teacher with privileged information but can induce a privilege illusion: the student learns privilege-dependent behavior it cannot reproduce from its inference-time context, yet behaves as if the training-time privileged information remained available, ultimately degrading performance. In this paper, we identify information asymmetry between the privileged teacher and the student at inference as the root cause of this failure in OPSD. To resolve this asymmetry, we propose Dual-Anchored Policy Distillation (DAPD), a unified framework with two levels of anchoring. Dual-Path Anchoring (DPA) introduces a self-conditioned bridge and aligns reference and rollout behavior along two matched-information paths, preventing privilege-dependent behavior from being transferred to the inference-time student. Dual-Source Anchoring (DSA) applies these paths in both reference-to-rollout and rollout-to-reference directions, reducing reliance on privileged reference guidance while preserving correctness supervision. Extensive experiments show that DAPD significantly alleviates privilege illusion, outperforming OPSD on Qwen3-4B by +2.00 points on average across tasks. Notably, its gains persist across scales, reaching +2.69 at 4B and +2.78 at 32B.
Community
DAPD aligns on-policy and reference behavior under matched information conditions.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2608.01735 in a model README.md to link it from this page.
Cite arxiv.org/abs/2608.01735 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2608.01735 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.