Large language models inevitably retain sensitive information, defined as inputs that may induce harmful generations, due to training on massive web corpora, raising concerns for privacy and safety. Existing machine unlearning methods primarily rely on retraining or aggressive fine-tuning, which are either computationally expensive or prone to degrading related knowledge and overall model utility. In this work, we reformulate machine unlearning as a precise knowledge re-mapping problem via model editing. We propose ZeroUnlearn, a few-shot unlearning framework. It overwrites sensitive inputs by mapping them to a neutral target state and removing their original representations. ZeroUnlearn enforces representational orthogonality through a multiplicative parameter update with a closed-form solution, enabling efficient and targeted unlearning. We further extend ZeroUnlearn to a gradient-based variant for multi-sample unlearning. Experiments demonstrate that our approach outperforms existing baselines while preserving general model utility. Our code is available at the github: <a href=\"https://github.com/XMUDeepLIT/ZeroUnlearn\" rel=\"nofollow\">https://github.com/XMUDeepLIT/ZeroUnlearn</a>.</p>\n","updatedAt":"2026-05-27T09:36:48.858Z","author":{"_id":"681ab9d3d7dbd87287875667","avatarUrl":"/avatars/1a2785d7a250c4988b1c1c5cc78e53fc.svg","fullname":"ChengyiYang","name":"ChengyiYang","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8833843469619751},"editors":["ChengyiYang"],"editorAvatarUrls":["/avatars/1a2785d7a250c4988b1c1c5cc78e53fc.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2605.18879","authors":[{"_id":"6a16bae1991d34bf20350052","name":"Yujie Lin","hidden":false},{"_id":"6a16bae1991d34bf20350053","name":"Chengyi Yang","hidden":false},{"_id":"6a16bae1991d34bf20350054","name":"Zhishang Xiang","hidden":false},{"_id":"6a16bae1991d34bf20350055","name":"Yiping Song","hidden":false},{"_id":"6a16bae1991d34bf20350056","name":"Jinsong Su","hidden":false}],"publishedAt":"2026-05-20T00:00:00.000Z","submittedOnDailyAt":"2026-05-27T00:00:00.000Z","title":"ZeroUnlearn: Few-Shot Knowledge Unlearning in Large Language Models","submittedOnDailyBy":{"_id":"681ab9d3d7dbd87287875667","avatarUrl":"/avatars/1a2785d7a250c4988b1c1c5cc78e53fc.svg","isPro":false,"fullname":"ChengyiYang","user":"ChengyiYang","type":"user","name":"ChengyiYang"},"summary":"Large language models inevitably retain sensitive information, defined as inputs that may induce harmful generations, due to training on massive web corpora, raising concerns for privacy and safety. Existing machine unlearning methods primarily rely on retraining or aggressive fine-tuning, which are either computationally expensive or prone to degrading related knowledge and overall model utility. In this work, we reformulate machine unlearning as a precise knowledge re-mapping problem via model editing. We propose ZeroUnlearn, a few-shot unlearning framework. It overwrites sensitive inputs by mapping them to a neutral target state and removing their original representations. ZeroUnlearn enforces representational orthogonality through a multiplicative parameter update with a closed-form solution, enabling efficient and targeted unlearning. We further extend ZeroUnlearn to a gradient-based variant for multi-sample unlearning. Experiments demonstrate that our approach outperforms existing baselines while preserving general model utility. Our code is available at the github: https://github.com/XMUDeepLIT/ZeroUnlearn.","upvotes":1,"discussionId":"6a16bae2991d34bf20350057","githubRepo":"https://github.com/XMUDeepLIT/ZeroUnlearn","githubRepoAddedBy":"user","ai_summary":"ZeroUnlearn addresses privacy concerns in large language models by reformulating machine unlearning as precise knowledge re-mapping through model editing, enabling efficient and targeted removal of sensitive information while preserving general model utility.","ai_keywords":["machine unlearning","model editing","sensitive information","representational orthogonality","multiplicative parameter update","closed-form solution","gradient-based variant","multi-sample unlearning"],"githubStars":2},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"681ab9d3d7dbd87287875667","avatarUrl":"/avatars/1a2785d7a250c4988b1c1c5cc78e53fc.svg","isPro":false,"fullname":"ChengyiYang","user":"ChengyiYang","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2605/2605.18879.md"}">
ZeroUnlearn: Few-Shot Knowledge Unlearning in Large Language Models
Abstract
ZeroUnlearn addresses privacy concerns in large language models by reformulating machine unlearning as precise knowledge re-mapping through model editing, enabling efficient and targeted removal of sensitive information while preserving general model utility.
AI-generated summary
Large language models inevitably retain sensitive information, defined as inputs that may induce harmful generations, due to training on massive web corpora, raising concerns for privacy and safety. Existing machine unlearning methods primarily rely on retraining or aggressive fine-tuning, which are either computationally expensive or prone to degrading related knowledge and overall model utility. In this work, we reformulate machine unlearning as a precise knowledge re-mapping problem via model editing. We propose ZeroUnlearn, a few-shot unlearning framework. It overwrites sensitive inputs by mapping them to a neutral target state and removing their original representations. ZeroUnlearn enforces representational orthogonality through a multiplicative parameter update with a closed-form solution, enabling efficient and targeted unlearning. We further extend ZeroUnlearn to a gradient-based variant for multi-sample unlearning. Experiments demonstrate that our approach outperforms existing baselines while preserving general model utility. Our code is available at the github: https://github.com/XMUDeepLIT/ZeroUnlearn.
Community
Large language models inevitably retain sensitive information, defined as inputs that may induce harmful generations, due to training on massive web corpora, raising concerns for privacy and safety. Existing machine unlearning methods primarily rely on retraining or aggressive fine-tuning, which are either computationally expensive or prone to degrading related knowledge and overall model utility. In this work, we reformulate machine unlearning as a precise knowledge re-mapping problem via model editing. We propose ZeroUnlearn, a few-shot unlearning framework. It overwrites sensitive inputs by mapping them to a neutral target state and removing their original representations. ZeroUnlearn enforces representational orthogonality through a multiplicative parameter update with a closed-form solution, enabling efficient and targeted unlearning. We further extend ZeroUnlearn to a gradient-based variant for multi-sample unlearning. Experiments demonstrate that our approach outperforms existing baselines while preserving general model utility. Our code is available at the github: https://github.com/XMUDeepLIT/ZeroUnlearn.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2605.18879 in a model README.md to link it from this page.
Cite arxiv.org/abs/2605.18879 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2605.18879 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.