Popular facts are memorised more deeply during pretraining and resist removal longer than rare ones, yet existing LLM unlearning methods apply uniform gradient pressure regardless of training-data frequency. We propose the AdaPop (Adaptive Popularity) method, which combines local token confidence with a per-fact popularity-dependent exponent derived from an external proxy (e.g., Wikidata sitelinks, LLM-as-Judge), and automates the forget-retain balance via a dual-ascent controller that adjusts the retain penalty each epoch. Across three model families and two benchmarks, AdaPop leaks ~5x less forgotten content than competing methods under paraphrased queries and ~1.6x less under adversarial reformulations. We support our analysis with internal metrics: under our method, forget-set hidden states move further from the pre-unlearning model's states than under other methods, while retain-set representations remain close.</p>\n","updatedAt":"2026-08-20T08:17:12.212Z","author":{"_id":"662f8d645c4db70c77a203b0","avatarUrl":"/avatars/72f9a3c39b3ba5114388d16a35524835.svg","fullname":"Elena Tutubalina","name":"tlenusik","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":7,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8573484420776367},"editors":["tlenusik"],"editorAvatarUrls":["/avatars/72f9a3c39b3ba5114388d16a35524835.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.14229","authors":[{"_id":"6a86b6c6db13816030683fd9","name":"Anna Borisiuk","hidden":false},{"_id":"6a86b6c6db13816030683fda","name":"Andrey Savchenko","hidden":false},{"_id":"6a86b6c6db13816030683fdb","name":"Alexander Panchenko","hidden":false},{"_id":"6a86b6c6db13816030683fdc","user":{"_id":"662f8d645c4db70c77a203b0","avatarUrl":"/avatars/72f9a3c39b3ba5114388d16a35524835.svg","isPro":false,"fullname":"Elena Tutubalina","user":"tlenusik","type":"user","name":"tlenusik"},"name":"Elena Tutubalina","status":"claimed_verified","statusLastChangedAt":"2026-08-20T08:45:04.904Z","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/662f8d645c4db70c77a203b0/HfCCyICcC1Ueyr9-tSvOr.jpeg"],"publishedAt":"2026-08-14T00:00:00.000Z","submittedOnDailyAt":"2026-08-20T00:00:00.000Z","title":"The More Popular, The Harder to Forget: Adaptive Popularity for LLM Unlearning","submittedOnDailyBy":{"_id":"662f8d645c4db70c77a203b0","avatarUrl":"/avatars/72f9a3c39b3ba5114388d16a35524835.svg","isPro":false,"fullname":"Elena Tutubalina","user":"tlenusik","type":"user","name":"tlenusik"},"summary":"Popular facts are memorised more deeply during pretraining and resist removal longer than rare ones, yet existing LLM unlearning methods apply uniform gradient pressure regardless of training-data frequency. We propose the AdaPop (Adaptive Popularity) method, which combines local token confidence with a per-fact popularity-dependent exponent derived from an external proxy (e.g., Wikidata sitelinks, LLM-as-Judge), and automates the forget-retain balance via a dual-ascent controller that adjusts the retain penalty each epoch. Across three model families and two benchmarks, AdaPop leaks ~5x less forgotten content than competing methods under paraphrased queries and ~1.6x less under adversarial reformulations. We support our analysis with internal metrics: under our method, forget-set hidden states move further from the pre-unlearning model's states than under other methods, while retain-set representations remain close.","upvotes":3,"discussionId":"6a86b6c6db13816030683fdd","ai_summary":"AdaPop adapts gradient pressure by fact popularity and automates forget-retain balance to reduce leakage of unlearned content.","ai_keywords":["AdaPop","local token confidence","per-fact popularity-dependent exponent","dual-ascent controller","forget-retain balance","hidden states"],"ai_summary_model":"thinkingmachines/Inkling-Small"},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"641b83837c21ab946bf50cc5","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/641b83837c21ab946bf50cc5/SW-yrQiNsg9N8MHsP-tZ9.jpeg","isPro":false,"fullname":"Sweetie Paws","user":"SwetieePawsss","type":"user"},{"_id":"662f8d645c4db70c77a203b0","avatarUrl":"/avatars/72f9a3c39b3ba5114388d16a35524835.svg","isPro":false,"fullname":"Elena Tutubalina","user":"tlenusik","type":"user"},{"_id":"60cd95ee15ecba5f2200304a","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/60cd95ee15ecba5f2200304a/3gMYeWm8wQO5KfqE5RmEe.jpeg","isPro":false,"fullname":"Alexey Dontsov","user":"therem","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.14229.md","query":{}}">
The More Popular, The Harder to Forget: Adaptive Popularity for LLM Unlearning
Abstract
AdaPop adapts gradient pressure by fact popularity and automates forget-retain balance to reduce leakage of unlearned content.
Popular facts are memorised more deeply during pretraining and resist removal longer than rare ones, yet existing LLM unlearning methods apply uniform gradient pressure regardless of training-data frequency. We propose the AdaPop (Adaptive Popularity) method, which combines local token confidence with a per-fact popularity-dependent exponent derived from an external proxy (e.g., Wikidata sitelinks, LLM-as-Judge), and automates the forget-retain balance via a dual-ascent controller that adjusts the retain penalty each epoch. Across three model families and two benchmarks, AdaPop leaks ~5x less forgotten content than competing methods under paraphrased queries and ~1.6x less under adversarial reformulations. We support our analysis with internal metrics: under our method, forget-set hidden states move further from the pre-unlearning model's states than under other methods, while retain-set representations remain close.
Community
Popular facts are memorised more deeply during pretraining and resist removal longer than rare ones, yet existing LLM unlearning methods apply uniform gradient pressure regardless of training-data frequency. We propose the AdaPop (Adaptive Popularity) method, which combines local token confidence with a per-fact popularity-dependent exponent derived from an external proxy (e.g., Wikidata sitelinks, LLM-as-Judge), and automates the forget-retain balance via a dual-ascent controller that adjusts the retain penalty each epoch. Across three model families and two benchmarks, AdaPop leaks ~5x less forgotten content than competing methods under paraphrased queries and ~1.6x less under adversarial reformulations. We support our analysis with internal metrics: under our method, forget-set hidden states move further from the pre-unlearning model's states than under other methods, while retain-set representations remain close.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2608.14229 in a model README.md to link it from this page.
Cite arxiv.org/abs/2608.14229 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2608.14229 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.