Real-world agent learning is often constrained by costly environment interactions, such as running time-consuming experiments or obtaining human feedback. In-context learning offers a highly sample-efficient way for agents to learn from their own interaction histories, but its gains disappear once that experience is removed from the context. Separately, context distillation provides a mechanism for internalizing contextual information into model weights. However, applying it to agents' interaction histories without sacrificing environment sample efficiency remains underexplored. We term this problem Experience Distillation and develop an implementation that requires no further environment interaction beyond the collected experience. Experiments on 749 curated software-engineering tasks and six text-adventure games show that it retains at least 64.8% of the gains from in-context learning across both domains, whereas direct supervised fine-tuning on the collected experience recovers only 3.8%. Compared with classical reinforcement-learning baselines, in-context learning from trial-and-error experience followed by Experience Distillation matches their performance with at least (9.6\\times) fewer environment samples.</p>\n","updatedAt":"2026-07-24T06:12:21.324Z","author":{"_id":"652e9c5774d1b0d7ff73d091","avatarUrl":"/avatars/a6d2098b3dde4a8b7488a193f0ecb776.svg","fullname":"Chenhui Gou","name":"gouc","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":6,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9354601502418518},"editors":["gouc"],"editorAvatarUrls":["/avatars/a6d2098b3dde4a8b7488a193f0ecb776.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2607.21051","authors":[{"_id":"6a6301ff2ee212ed0e2a1597","user":{"_id":"652e9c5774d1b0d7ff73d091","avatarUrl":"/avatars/a6d2098b3dde4a8b7488a193f0ecb776.svg","isPro":true,"fullname":"Chenhui Gou","user":"gouc","type":"user","name":"gouc"},"name":"Chenhui Gou","status":"claimed_verified","statusLastChangedAt":"2026-07-24T08:45:04.326Z","hidden":false},{"_id":"6a6301ff2ee212ed0e2a1598","name":"Haoqin Tu","hidden":false},{"_id":"6a6301ff2ee212ed0e2a1599","name":"Yunhao Fang","hidden":false},{"_id":"6a6301ff2ee212ed0e2a159a","name":"Jianfei Cai","hidden":false},{"_id":"6a6301ff2ee212ed0e2a159b","name":"Hamid Rezatofighi","hidden":false}],"publishedAt":"2026-07-23T00:00:00.000Z","submittedOnDailyAt":"2026-07-24T00:00:00.000Z","title":"Sample-Efficient Learning from Agent Experience","submittedOnDailyBy":{"_id":"652e9c5774d1b0d7ff73d091","avatarUrl":"/avatars/a6d2098b3dde4a8b7488a193f0ecb776.svg","isPro":true,"fullname":"Chenhui Gou","user":"gouc","type":"user","name":"gouc"},"summary":"Real-world agent learning is often constrained by costly environment interactions, such as running time-consuming experiments or obtaining human feedback. In-context learning offers a highly sample-efficient way for agents to learn from their own interaction histories, but its gains disappear once that experience is removed from the context. Separately, context distillation provides a mechanism for internalizing contextual information into model weights. However, applying it to agents' interaction histories without sacrificing environment sample efficiency remains underexplored. We term this problem Experience Distillation and develop an implementation that requires no further environment interaction beyond the collected experience. Experiments on 749 curated software-engineering tasks and six text-adventure games show that it retains at least 64.8\\% of the gains from in-context learning across both domains, whereas direct supervised fine-tuning on the collected experience recovers only 3.8\\%. Compared with classical reinforcement-learning baselines, in-context learning from trial-and-error experience followed by Experience Distillation matches their performance with at least \\(9.6\\times\\) fewer environment samples.","upvotes":7,"discussionId":"6a6302002ee212ed0e2a159c"},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"652e9c5774d1b0d7ff73d091","avatarUrl":"/avatars/a6d2098b3dde4a8b7488a193f0ecb776.svg","isPro":true,"fullname":"Chenhui Gou","user":"gouc","type":"user"},{"_id":"67f69fbb76f86fce3c9050c3","avatarUrl":"/avatars/c3aa74f44a625f6a486a5d4a7cfa9fdb.svg","isPro":false,"fullname":"Jet Xia","user":"JetttX","type":"user"},{"_id":"64e4446756b920ef00f2031c","avatarUrl":"/avatars/86305d1eac3917bf9859ac0ddbb4e845.svg","isPro":false,"fullname":"Zicheng Duan","user":"ZichengD","type":"user"},{"_id":"63ac5701c21e60a3e9b58aa7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63ac5701c21e60a3e9b58aa7/g6EX7diOpuA94R2ab-rZC.png","isPro":true,"fullname":"Dipankar Sarkar","user":"dipankarsarkar","type":"user"},{"_id":"663bbb61ec1aafe3d6c05558","avatarUrl":"/avatars/b2f593d0ae0adbaad9a7a99b490e3a2b.svg","isPro":false,"fullname":"Ziyu Ma","user":"poiuytrewq123","type":"user"},{"_id":"697c8b15a7f796854ef333c4","avatarUrl":"/avatars/94de3a736fac914944f1b57609e3819a.svg","isPro":false,"fullname":"Joel Wang","user":"joelhenwang","type":"user"},{"_id":"69a3fc75a022769cc825d47d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/_lZ52_J42oOLitKVgFgWq.jpeg","isPro":false,"fullname":"Zhou Yutong","user":"WYATTMOORE45","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"query":{}}">
Sample-Efficient Learning from Agent Experience
Abstract
Real-world agent learning is often constrained by costly environment interactions, such as running time-consuming experiments or obtaining human feedback. In-context learning offers a highly sample-efficient way for agents to learn from their own interaction histories, but its gains disappear once that experience is removed from the context. Separately, context distillation provides a mechanism for internalizing contextual information into model weights. However, applying it to agents' interaction histories without sacrificing environment sample efficiency remains underexplored. We term this problem Experience Distillation and develop an implementation that requires no further environment interaction beyond the collected experience. Experiments on 749 curated software-engineering tasks and six text-adventure games show that it retains at least 64.8\% of the gains from in-context learning across both domains, whereas direct supervised fine-tuning on the collected experience recovers only 3.8\%. Compared with classical reinforcement-learning baselines, in-context learning from trial-and-error experience followed by Experience Distillation matches their performance with at least \(9.6\times\) fewer environment samples.
Community
Real-world agent learning is often constrained by costly environment interactions, such as running time-consuming experiments or obtaining human feedback. In-context learning offers a highly sample-efficient way for agents to learn from their own interaction histories, but its gains disappear once that experience is removed from the context. Separately, context distillation provides a mechanism for internalizing contextual information into model weights. However, applying it to agents' interaction histories without sacrificing environment sample efficiency remains underexplored. We term this problem Experience Distillation and develop an implementation that requires no further environment interaction beyond the collected experience. Experiments on 749 curated software-engineering tasks and six text-adventure games show that it retains at least 64.8% of the gains from in-context learning across both domains, whereas direct supervised fine-tuning on the collected experience recovers only 3.8%. Compared with classical reinforcement-learning baselines, in-context learning from trial-and-error experience followed by Experience Distillation matches their performance with at least (9.6\times) fewer environment samples.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2607.21051 in a model README.md to link it from this page.
Cite arxiv.org/abs/2607.21051 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2607.21051 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.