Rethinking Continual Experience Internalization for Self-Evolving LLM Agents</p>\n","updatedAt":"2026-06-05T02:45:52.989Z","author":{"_id":"64b7df742f5a966b973e25f7","avatarUrl":"/avatars/e24e7769188d441317b3b7d10ef8fd60.svg","fullname":"Wenkai Yang","name":"Keven16","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":15,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.6862868070602417},"editors":["Keven16"],"editorAvatarUrls":["/avatars/e24e7769188d441317b3b7d10ef8fd60.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.04703","authors":[{"_id":"6a20fa4615100c5272a8478a","user":{"_id":"664c94f71959997352fc1946","avatarUrl":"/avatars/1622bea455771298658578fab24ecee7.svg","isPro":false,"fullname":"Jingwen Chen","user":"cjw259wen","type":"user","name":"cjw259wen"},"name":"Jingwen Chen","status":"claimed_verified","statusLastChangedAt":"2026-06-04T12:39:24.442Z","hidden":false},{"_id":"6a20fa4615100c5272a8478b","name":"Wenkai Yang","hidden":false},{"_id":"6a20fa4615100c5272a8478c","name":"Shengda Fan","hidden":false},{"_id":"6a20fa4615100c5272a8478d","name":"Wenbo Nie","hidden":false},{"_id":"6a20fa4615100c5272a8478e","name":"Chenxing Sun","hidden":false},{"_id":"6a20fa4615100c5272a8478f","name":"Shaodong Zheng","hidden":false},{"_id":"6a20fa4615100c5272a84790","name":"Yangen Hu","hidden":false},{"_id":"6a20fa4615100c5272a84791","name":"Lu Pan","hidden":false},{"_id":"6a20fa4615100c5272a84792","name":"Ke Zeng","hidden":false},{"_id":"6a20fa4615100c5272a84793","name":"Yankai Lin","hidden":false}],"publishedAt":"2026-06-03T00:00:00.000Z","submittedOnDailyAt":"2026-06-05T00:00:00.000Z","title":"Rethinking Continual Experience Internalization for Self-Evolving LLM Agents","submittedOnDailyBy":{"_id":"64b7df742f5a966b973e25f7","avatarUrl":"/avatars/e24e7769188d441317b3b7d10ef8fd60.svg","isPro":false,"fullname":"Wenkai Yang","user":"Keven16","type":"user","name":"Keven16"},"summary":"Experience internalization converts contextual experience from past interactions into reusable parametric capability, offering a promising path toward continual learning in large language models (LLMs). While prior work has predominantly focused on single-iteration transfer, we discover that under multi-iteration experience learning, existing methods suffer from a progressive capability collapse rather than compounding improvement. We systematically examine this failure through three vital dimensions of experience internalization: (1) Experience Granularity: We find that principle-level experience is more durable than instance-level experience, as it effectively abstracts transferable strategies away from trajectory-specific details. (2) Experience Injection Pattern: Our analysis reveals that step-wise injection significantly outperforms global injection by aligning experience with intermediate decision states, a property that is critical for long-horizon tool use. (3) Internalization Regime: We demonstrate that off-policy context-distillation on high-quality teacher trajectories provides a substantially more stable training signal than on-policy context-distillation, which is inherently limited by local corrections on student-induced flawed states. Together, these insights yield a simple yet robust recipe for stable and sustainable experience internalization, providing concrete guidance for engineering self-evolving and continually learning LLMs.","upvotes":15,"discussionId":"6a20fa4615100c5272a84794","githubRepo":"https://github.com/RUCBM/ExpInternalization","githubRepoAddedBy":"user","ai_summary":"Experience internalization enables continual learning in large language models by converting past interactions into reusable capabilities, with key findings on experience granularity, injection patterns, and internalization regimes for stable learning.","ai_keywords":["experience internalization","continual learning","large language models","single-iteration transfer","multi-iteration experience learning","capability collapse","experience granularity","instance-level experience","principle-level experience","experience injection pattern","step-wise injection","global injection","internalization regime","off-policy context-distillation","on-policy context-distillation","teacher trajectories","student-induced flawed states"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","githubStars":0},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"66f6c6aa10cd972c8653e78e","avatarUrl":"/avatars/a59a8e8f867c67f86d4eeea9e6f3065a.svg","isPro":false,"fullname":"shuang e","user":"NNNNNr","type":"user"},{"_id":"64b7df742f5a966b973e25f7","avatarUrl":"/avatars/e24e7769188d441317b3b7d10ef8fd60.svg","isPro":false,"fullname":"Wenkai Yang","user":"Keven16","type":"user"},{"_id":"66344e79b7861e2db7716c34","avatarUrl":"/avatars/3db5b84035f1b99f0af809b8a4b4d014.svg","isPro":false,"fullname":"Chenxing Sun","user":"ChenxingSun","type":"user"},{"_id":"66beae55c55655c71507adc4","avatarUrl":"/avatars/56e63dba6b55cb03734e70c3d1199874.svg","isPro":false,"fullname":"AnIdealRing","user":"SmartDazi","type":"user"},{"_id":"69c5e8ce7501fc2ff97103a8","avatarUrl":"/avatars/f8b5c9e8fc66ad24a9cf8837673c044f.svg","isPro":false,"fullname":"Shadow Zheng","user":"feiwuu638","type":"user"},{"_id":"68e7b092c1a1d4eb80a56e47","avatarUrl":"/avatars/b47067cfaa2febb1c4ccde88b682e20a.svg","isPro":false,"fullname":"yaoyou fan","user":"yyhhdd","type":"user"},{"_id":"698ab2ebc9804eab58756f66","avatarUrl":"/avatars/797aa01a039a42671b8140c7742c71a5.svg","isPro":false,"fullname":"ShuqiYe","user":"ShuqiYe","type":"user"},{"_id":"64bb937d8496ee0fb6cac9aa","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64bb937d8496ee0fb6cac9aa/oFkKNxaMrd3wAciwP4Lu5.png","isPro":false,"fullname":"YijuGuo","user":"YijuGuo","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"664c94f71959997352fc1946","avatarUrl":"/avatars/1622bea455771298658578fab24ecee7.svg","isPro":false,"fullname":"Jingwen Chen","user":"cjw259wen","type":"user"},{"_id":"67dbc13e47f06bcb49de28a8","avatarUrl":"/avatars/758acafe51e076c7b5f281b1d860db45.svg","isPro":false,"fullname":"xiaocongda","user":"congda","type":"user"},{"_id":"674721f0fcb9481d1e58a081","avatarUrl":"/avatars/ea4087699f1966127719f64a5d80042a.svg","isPro":false,"fullname":"李明业","user":"Mingye-Li","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.04703.md"}">

Get this paper in your agent:

Don't have the latest CLI?

Cite arxiv.org/abs/2606.04703 in a model README.md to link it from this page.

Cite arxiv.org/abs/2606.04703 in a dataset README.md to link it from this page.

Cite arxiv.org/abs/2606.04703 in a Space README.md to link it from this page.