Hugging Face Daily Papers · May 21, 2026 · 3 min read

It Takes Two: Complementary Self-Distillation for Contextual Integrity in LLMs

Mirrored from Hugging Face Daily Papers for archival readability. Support the source by reading on the original site.

Like Read original ↗

We introduce SelfCI, a complementary self-distillation framework that aligns LLMs with Contextual Integrity (CI) by jointly optimizing utility-preserving and privacy-preserving objectives, achieving stronger privacy–utility trade-offs than existing RL-based approaches.<br><a href=\"https://cdn-uploads.huggingface.co/production/uploads/66d30f5fad293ffc4b7672bc/Lbsusr5nG1xeNYP2KmMEQ.png\" rel=\"nofollow\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/66d30f5fad293ffc4b7672bc/Lbsusr5nG1xeNYP2KmMEQ.png\" alt=\"main_fig\"></a></p>\n","updatedAt":"2026-05-21T03:13:43.508Z","author":{"_id":"66d30f5fad293ffc4b7672bc","avatarUrl":"/avatars/6f164d813b947940a088820f8fd4dbe8.svg","fullname":"Woongyeong Yeo","name":"wgcyeo","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":10,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.832444965839386},"editors":["wgcyeo"],"editorAvatarUrls":["/avatars/6f164d813b947940a088820f8fd4dbe8.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2605.20258","authors":[{"_id":"6a0e76dd164dbbc68a26c4d5","name":"Sangwoo Park","hidden":false},{"_id":"6a0e76dd164dbbc68a26c4d6","name":"Woongyeong Yeo","hidden":false},{"_id":"6a0e76dd164dbbc68a26c4d7","name":"Seanie Lee","hidden":false},{"_id":"6a0e76dd164dbbc68a26c4d8","name":"Yumin Choi","hidden":false},{"_id":"6a0e76dd164dbbc68a26c4d9","name":"Hyomin Lee","hidden":false},{"_id":"6a0e76dd164dbbc68a26c4da","name":"Kangsan Kim","hidden":false},{"_id":"6a0e76dd164dbbc68a26c4db","name":"Jinheon Baek","hidden":false},{"_id":"6a0e76dd164dbbc68a26c4dc","name":"Seong Joon Oh","hidden":false},{"_id":"6a0e76dd164dbbc68a26c4dd","name":"Sung Ju Hwang","hidden":false}],"publishedAt":"2026-05-18T00:00:00.000Z","submittedOnDailyAt":"2026-05-21T00:00:00.000Z","title":"It Takes Two: Complementary Self-Distillation for Contextual Integrity in LLMs","submittedOnDailyBy":{"_id":"66d30f5fad293ffc4b7672bc","avatarUrl":"/avatars/6f164d813b947940a088820f8fd4dbe8.svg","isPro":false,"fullname":"Woongyeong Yeo","user":"wgcyeo","type":"user","name":"wgcyeo"},"summary":"Contextual Integrity (CI) defines privacy not merely as keeping information hidden, but as governing information flows according to the norms of a given context. As large language models are increasingly deployed as personal agents handling sensitive workflows, adhering to CI becomes critical. However, even frontier models remain unreliable in making disclosure decisions, and existing mitigation strategies often degrade underlying task performance. To overcome this privacy-utility trade-off, we propose SELFCI, a complementary self-distillation framework that decouples information suppression from task resolution. SELFCI jointly optimizes two independent reverse KL divergences over distinct teacher distributions derived from feedback: one encourages preserving task-relevant information for utility, while the other enforces minimal and appropriate disclosure. This complementary formulation induces a Product-of-Experts (PoE) target, aligning the policy with the intersection of capability and privacy requirements. Empirical evaluations demonstrate that SELFCI, without relying on costly external supervision, consistently outperforms competitive baselines such as online reinforcement learning algorithms (e.g., GRPO). These trends further extend to out-of-domain settings involving agentic workflows and accumulated private context, suggesting that SELFCI provides a practical path toward CI alignment.","upvotes":25,"discussionId":"6a0e76de164dbbc68a26c4de","githubRepo":"https://github.com/sw-programmer/SelfCI","githubRepoAddedBy":"user","ai_summary":"SELFCI is a self-distillation framework that separates information suppression from task resolution to achieve better privacy-utility balance in large language models without external supervision.","ai_keywords":["contextual integrity","large language models","information flows","self-distillation","reverse KL divergence","Product-of-Experts","policy optimization","disclosure decisions","task resolution","privacy-utility trade-off"],"githubStars":1,"organization":{"_id":"6475760c33192631bad2bb38","name":"kaist-ai","fullname":"KAIST AI","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/6469949654873f0043b09c22/aaZFiyXe1qR-Dmy_xq67m.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"63036b6c5c70c21d0ea79d48","avatarUrl":"/avatars/a7eb03f5cbd4eaa09fe807bbed8bc0f7.svg","isPro":false,"fullname":"Jinheon Baek","user":"jinheon","type":"user"},{"_id":"638716c14e00d7fc0902fef4","avatarUrl":"/avatars/5fa8152f8c0e4e600d1a64802c3e0103.svg","isPro":false,"fullname":"Sangwoo Park","user":"Sangsang","type":"user"},{"_id":"65f06c8356cb8a32b41baf83","avatarUrl":"/avatars/ac66cc63f3abade4a859f7bf9357682a.svg","isPro":false,"fullname":"Jiongdao Jin","user":"jiongdao","type":"user"},{"_id":"64b5457af249713053c736c5","avatarUrl":"/avatars/84cd17e11f20aee404f7ffadf659cd6f.svg","isPro":false,"fullname":"Yukyeong Lee","user":"leee99","type":"user"},{"_id":"64b74920fe6a108d03fed767","avatarUrl":"/avatars/a2c05b809c36fa5fab8e1a43b3e67051.svg","isPro":false,"fullname":"Minki Kang","user":"Nardien","type":"user"},{"_id":"695b4e723631aa29113d7b34","avatarUrl":"/avatars/0bf9e44919744f9a067573d9d14c05c8.svg","isPro":false,"fullname":"Ji","user":"543family","type":"user"},{"_id":"65caff7e024c6320f4384130","avatarUrl":"/avatars/67f7c20372bfcc0c1554fb8c9284653a.svg","isPro":false,"fullname":"Suji Kim","user":"SujiKim","type":"user"},{"_id":"654738a75cd5692b3ac5d5f0","avatarUrl":"/avatars/77262c6c89193962ae5b21136dad2370.svg","isPro":false,"fullname":"Byeong Hun Bang","user":"bangpro","type":"user"},{"_id":"66ed7737f2f27a5dfd81ef09","avatarUrl":"/avatars/f45eea356e92ac7b3db23c2c92dec9fa.svg","isPro":false,"fullname":"Kangsan Kim","user":"KangsanKim71","type":"user"},{"_id":"65c98aed8974af76544491ce","avatarUrl":"/avatars/6aab28f64bafeb62ed3c33565b268601.svg","isPro":false,"fullname":"KIMTAEIL","user":"7stars-commander","type":"user"},{"_id":"64e5a1cd4c20016ec9020ec8","avatarUrl":"/avatars/d7ffe7fbbe39c0a013375357457c57b3.svg","isPro":false,"fullname":"Soyeong","user":"starsuzi","type":"user"},{"_id":"65633c5e84a9fbe322f87d81","avatarUrl":"/avatars/7233a555b43c669847a950ce5697c92c.svg","isPro":false,"fullname":"DongkiKim","user":"DongkiKim","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"6475760c33192631bad2bb38","name":"kaist-ai","fullname":"KAIST AI","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/6469949654873f0043b09c22/aaZFiyXe1qR-Dmy_xq67m.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2605/2605.20258.md"}">

Papers

arxiv:2605.20258

It Takes Two: Complementary Self-Distillation for Contextual Integrity in LLMs

Published on May 18

· Submitted by

Woongyeong Yeo on May 21

KAIST AI

Upvote

Authors:

Abstract

SELFCI is a self-distillation framework that separates information suppression from task resolution to achieve better privacy-utility balance in large language models without external supervision.

AI-generated summary

Contextual Integrity (CI) defines privacy not merely as keeping information hidden, but as governing information flows according to the norms of a given context. As large language models are increasingly deployed as personal agents handling sensitive workflows, adhering to CI becomes critical. However, even frontier models remain unreliable in making disclosure decisions, and existing mitigation strategies often degrade underlying task performance. To overcome this privacy-utility trade-off, we propose SELFCI, a complementary self-distillation framework that decouples information suppression from task resolution. SELFCI jointly optimizes two independent reverse KL divergences over distinct teacher distributions derived from feedback: one encourages preserving task-relevant information for utility, while the other enforces minimal and appropriate disclosure. This complementary formulation induces a Product-of-Experts (PoE) target, aligning the policy with the intersection of capability and privacy requirements. Empirical evaluations demonstrate that SELFCI, without relying on costly external supervision, consistently outperforms competitive baselines such as online reinforcement learning algorithms (e.g., GRPO). These trends further extend to out-of-domain settings involving agentic workflows and accumulated private context, suggesting that SELFCI provides a practical path toward CI alignment.