This paper studies self-evolving LLM agents that improve by updating external harnesses, and separates harness-evolution from base task-solving capability along two dimensions: harness-updating, which writes useful persistent updates, and harness-benefit, which measures whether agents can benefit from those updates in future tasks.</p>\n<p>The key findings are twofold. First, stronger base models are not necessarily better harness updaters: evolvers across capability tiers yield surprisingly similar gains, with even the smaller Qwen3.5-9B evolver matching much stronger models such as Claude Opus 4.6. Second, harness-benefit is non-monotonic: weak models benefit little, mid-tier models benefit the most, and strong models benefit less than mid-tier models. The paper further shows that weak models often fail to activate relevant harness artifacts or follow them faithfully.</p>\n<p>Overall, the paper suggests that the main bottleneck in self-evolving agents may be less about using the strongest evolver and more about enabling agents to invoke and follow updated harnesses effectively.</p>\n<p>Our code is publicly available at <a href=\"https://github.com/A-EVO-Lab/a-evolve/tree/release/harness-evolution\" rel=\"nofollow\">https://github.com/A-EVO-Lab/a-evolve/tree/release/harness-evolution</a>.</p>\n","updatedAt":"2026-06-01T20:54:19.745Z","author":{"_id":"65f8ae0f6c02ff2f6d772f7e","avatarUrl":"/avatars/cb3798f4a7f55f928ed2f5ead0407d36.svg","fullname":"Minhua Lin","name":"ventr1c","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9169556498527527},"editors":["ventr1c"],"editorAvatarUrls":["/avatars/cb3798f4a7f55f928ed2f5ead0407d36.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2605.30621","authors":[{"_id":"6a1dee69808ddbc3c7d43b01","name":"Minhua Lin","hidden":false},{"_id":"6a1dee69808ddbc3c7d43b02","name":"Juncheng Wu","hidden":false},{"_id":"6a1dee69808ddbc3c7d43b03","name":"Zijun Wang","hidden":false},{"_id":"6a1dee69808ddbc3c7d43b04","name":"Zhan Shi","hidden":false},{"_id":"6a1dee69808ddbc3c7d43b05","name":"Yisi Sang","hidden":false},{"_id":"6a1dee69808ddbc3c7d43b06","name":"Bing He","hidden":false},{"_id":"6a1dee69808ddbc3c7d43b07","name":"Zewen Liu","hidden":false},{"_id":"6a1dee69808ddbc3c7d43b08","name":"Tianxin Wei","hidden":false},{"_id":"6a1dee69808ddbc3c7d43b09","name":"Zongyu Wu","hidden":false},{"_id":"6a1dee69808ddbc3c7d43b0a","name":"Zhiwei Zhang","hidden":false},{"_id":"6a1dee69808ddbc3c7d43b0b","name":"Dakuo Wang","hidden":false},{"_id":"6a1dee69808ddbc3c7d43b0c","name":"Xiang Zhang","hidden":false},{"_id":"6a1dee69808ddbc3c7d43b0d","name":"Benoit Dumoulin","hidden":false},{"_id":"6a1dee69808ddbc3c7d43b0e","name":"Cihang Xie","hidden":false},{"_id":"6a1dee69808ddbc3c7d43b0f","name":"Yuyin Zhou","hidden":false},{"_id":"6a1dee69808ddbc3c7d43b10","name":"Suhang Wang","hidden":false},{"_id":"6a1dee69808ddbc3c7d43b11","name":"Hanqing Lu","hidden":false}],"publishedAt":"2026-05-28T00:00:00.000Z","submittedOnDailyAt":"2026-06-01T00:00:00.000Z","title":"Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents","submittedOnDailyBy":{"_id":"65f8ae0f6c02ff2f6d772f7e","avatarUrl":"/avatars/cb3798f4a7f55f928ed2f5ead0407d36.svg","isPro":false,"fullname":"Minhua Lin","user":"ventr1c","type":"user","name":"ventr1c"},"summary":"LLM agents are increasingly deployed as systems built around editable external harnesses, including prompts, skills, memories and tools, that shape task execution without changing model parameters. Harness self-evolution adapts such agents by updating these harnesses from execution evidence. Yet it remains unclear whether a model's base capability in task-solving predicts its capabilities in harness self-evolution: which models produce useful harness updates, and which actually benefit from them? We analyze two harness self-evolution capabilities: (i) harness-updating, the capability to produce useful persistent harness updates from execution evidence; (ii) harness-benefit, the capability to benefit from updated harnesses during task solving. Our analysis reveals two findings. First, harness-updating is flat in base capability: models from different capability tiers produce harness updates that lead to surprisingly similar gains; even Qwen3.5-9B's updates yield gains comparable to those of Claude Opus~4.6. Second, harness-benefit is non-monotonic in base capability: weak-tier models benefit little from updated harnesses, mid-tier models benefit most, and strong-tier models benefit less than mid-tier. We trace low gains at the weak tier to two failure modes: weak-tier models may fail to activate relevant harness artifacts, or activate them but fail to follow them faithfully. These findings suggest investing capability budget in the task-solving agent rather than the evolver, and targeting harness invocation and long-horizon instruction following in agent training. Our source code is publicly available at https://github.com/A-EVO-Lab/a-evolve/tree/release/harness-evolution.","upvotes":3,"discussionId":"6a1dee69808ddbc3c7d43b12","ai_summary":"Research reveals that harness self-evolution capabilities in LLM agents show unexpected patterns: harness-updating effectiveness is consistent across model capabilities, while harness-benefit follows a non-monotonic trend with mid-tier models performing best.","ai_keywords":["LLM agents","harness self-evolution","task-solving","model parameters","base capability","harness-updating","harness-benefit","execution evidence","model training","agent training"],"organization":{"_id":"5ffdfbadbba2ae614d771970","name":"amazon","fullname":"Amazon","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/66f19ed428ae41c20c470792/8y7msN6A6W82LdQhQd85a.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"660026b7573abbcdb975a34f","avatarUrl":"/avatars/93defd0e6274cfe8f124220c59ec2bed.svg","isPro":false,"fullname":"Juncheng Wu","user":"Chtholly17","type":"user"},{"_id":"65f8ae0f6c02ff2f6d772f7e","avatarUrl":"/avatars/cb3798f4a7f55f928ed2f5ead0407d36.svg","isPro":false,"fullname":"Minhua Lin","user":"ventr1c","type":"user"},{"_id":"698391f7c79652c087ecd076","avatarUrl":"/avatars/2ec759f1f85486248b3da09bbc0f7d41.svg","isPro":false,"fullname":"Hanqing Lu","user":"HenryLuAI","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"5ffdfbadbba2ae614d771970","name":"amazon","fullname":"Amazon","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/66f19ed428ae41c20c470792/8y7msN6A6W82LdQhQd85a.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2605/2605.30621.md"}">

Get this paper in your agent:

Don't have the latest CLI?

Cite arxiv.org/abs/2605.30621 in a model README.md to link it from this page.

Cite arxiv.org/abs/2605.30621 in a dataset README.md to link it from this page.

Cite arxiv.org/abs/2605.30621 in a Space README.md to link it from this page.

Add this paper to a collection to link it from this page.