<a href=\"https://cdn-uploads.huggingface.co/production/uploads/658ad59e304552ba0c034d35/kFQsSoG8z_OifjevytKSk.png\" rel=\"nofollow\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/658ad59e304552ba0c034d35/kFQsSoG8z_OifjevytKSk.png\" alt=\"image\"></a></p>\n","updatedAt":"2026-07-13T02:17:12.143Z","author":{"_id":"658ad59e304552ba0c034d35","avatarUrl":"/avatars/ab0c9978b774e68b4d63eef8cca4417c.svg","fullname":"Lu Dai","name":"stellaludai","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.35246026515960693},"editors":["stellaludai"],"editorAvatarUrls":["/avatars/ab0c9978b774e68b4d63eef8cca4417c.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2607.08393","authors":[{"_id":"6a51c757a9d74d6e65bbc7f1","name":"Lu Dai","hidden":false},{"_id":"6a51c757a9d74d6e65bbc7f2","name":"Ziyang Rao","hidden":false},{"_id":"6a51c757a9d74d6e65bbc7f3","name":"Yili Wang","hidden":false},{"_id":"6a51c757a9d74d6e65bbc7f4","name":"Hanqing Wang","hidden":false},{"_id":"6a51c757a9d74d6e65bbc7f5","name":"Hao Liu","hidden":false},{"_id":"6a51c757a9d74d6e65bbc7f6","name":"Hui Xiong","hidden":false}],"publishedAt":"2026-07-09T12:17:28.000Z","submittedOnDailyAt":"2026-07-13T00:00:00.000Z","title":"Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning","submittedOnDailyBy":{"_id":"658ad59e304552ba0c034d35","avatarUrl":"/avatars/ab0c9978b774e68b4d63eef8cca4417c.svg","isPro":false,"fullname":"Lu Dai","user":"stellaludai","type":"user","name":"stellaludai"},"summary":"Fine-tuning LLMs to inject new knowledge faces a critical challenge: LLMs can quickly memorize new facts, yet fail to use them for downstream reasoning tasks. We formalize this failure as the \\textbf{Knowing--Using Gap}, characterized by an accuracy gap and a temporal lag between memorization and generalization. To understand this phenomenon, we fine-tune LLMs with unseen knowledge and monitor the spatial permeation dynamics of the knowledge internally using a novel intervention technique called self-patching. Self-patching identifies activation locations where relocating representations substantially improves failed generalization cases. These results are consistent with a knowledge-circuit misalignment hypothesis: memorized representations can exist internally but may not be routed to computation-effective layers. To demonstrate the practicality of this diagnostic finding, we design a simple heuristic strategy which recovers 58--75\\% of the oracle headroom in generalization failure. Experiments are done cross-domain for the robustness of this finding.","upvotes":1,"discussionId":"6a51c757a9d74d6e65bbc7f7","organization":{"_id":"63355133edc1a61aecf74b0e","name":"HKUST","fullname":"HKUST","avatar":"https://www.gravatar.com/avatar/4a4318de793d2c187cb6f312e9d0e7bc?d=retro&size=100"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"658ad59e304552ba0c034d35","avatarUrl":"/avatars/ab0c9978b774e68b4d63eef8cca4417c.svg","isPro":false,"fullname":"Lu Dai","user":"stellaludai","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"63355133edc1a61aecf74b0e","name":"HKUST","fullname":"HKUST","avatar":"https://www.gravatar.com/avatar/4a4318de793d2c187cb6f312e9d0e7bc?d=retro&size=100"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2607/2607.08393.md","query":{}}">
Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning
Published on Jul 9
· Submitted by Lu Dai on Jul 13 Abstract
Fine-tuning LLMs to inject new knowledge faces a critical challenge: LLMs can quickly memorize new facts, yet fail to use them for downstream reasoning tasks. We formalize this failure as the \textbf{Knowing--Using Gap}, characterized by an accuracy gap and a temporal lag between memorization and generalization. To understand this phenomenon, we fine-tune LLMs with unseen knowledge and monitor the spatial permeation dynamics of the knowledge internally using a novel intervention technique called self-patching. Self-patching identifies activation locations where relocating representations substantially improves failed generalization cases. These results are consistent with a knowledge-circuit misalignment hypothesis: memorized representations can exist internally but may not be routed to computation-effective layers. To demonstrate the practicality of this diagnostic finding, we design a simple heuristic strategy which recovers 58--75\% of the oracle headroom in generalization failure. Experiments are done cross-domain for the robustness of this finding.
Community
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2607.08393 in a model README.md to link it from this page.
Cite arxiv.org/abs/2607.08393 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2607.08393 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.