To achieve a vision for LLM-based long-lifecycle agents, we present a general framework of end-to-end RL, our proof-of-concept implementations, and some promising empirical results.</p>\n<p>GitHub: <a href=\"https://github.com/agentscope-ai/Trinity-RFT/tree/research/cod/examples/research_cod\" rel=\"nofollow\">https://github.com/agentscope-ai/Trinity-RFT/tree/research/cod/examples/research_cod</a></p>\n","updatedAt":"2026-06-23T03:03:08.318Z","author":{"_id":"6576f9f4654561a1b345610b","avatarUrl":"/avatars/f801f551640caa70368fcc26a0f51d27.svg","fullname":"Yanxi Chen","name":"yanxi-chen","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8028077483177185},"editors":["yanxi-chen"],"editorAvatarUrls":["/avatars/f801f551640caa70368fcc26a0f51d27.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.20002","authors":[{"_id":"6a39f3f0fdcd3514343bb501","user":{"_id":"6576f9f4654561a1b345610b","avatarUrl":"/avatars/f801f551640caa70368fcc26a0f51d27.svg","isPro":false,"fullname":"Yanxi Chen","user":"yanxi-chen","type":"user","name":"yanxi-chen"},"name":"Yanxi Chen","status":"claimed_verified","statusLastChangedAt":"2026-06-23T13:56:52.948Z","hidden":false},{"_id":"6a39f3f0fdcd3514343bb502","name":"Weijie Shi","hidden":false},{"_id":"6a39f3f0fdcd3514343bb503","name":"Yuexiang Xie","hidden":false},{"_id":"6a39f3f0fdcd3514343bb504","name":"Boyi Hu","hidden":false},{"_id":"6a39f3f0fdcd3514343bb505","name":"Yaliang Li","hidden":false},{"_id":"6a39f3f0fdcd3514343bb506","name":"Bolin Ding","hidden":false},{"_id":"6a39f3f0fdcd3514343bb507","name":"Jingren Zhou","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/6576f9f4654561a1b345610b/3tDyvFLGC4wvgmAm5YZB9.png","https://cdn-uploads.huggingface.co/production/uploads/6576f9f4654561a1b345610b/XSGkkMkQX6fo2Cgz8fXe7.png"],"publishedAt":"2026-06-18T00:00:00.000Z","submittedOnDailyAt":"2026-06-23T00:00:00.000Z","title":"Connect the Dots: Training LLMs for Long-Lifecycle Agents with Cross-Domain Generalization Via Reinforcement Learning","submittedOnDailyBy":{"_id":"6576f9f4654561a1b345610b","avatarUrl":"/avatars/f801f551640caa70368fcc26a0f51d27.svg","isPro":false,"fullname":"Yanxi Chen","user":"yanxi-chen","type":"user","name":"yanxi-chen"},"summary":"This work presents a general framework for training large language models (LLMs) to \"Connect the Dots\" (CoD), a meta-capability required by long-lifecycle agents: as an LLM-based AI agent gets deployed in an environment, it solves a long sequence of tasks while continuously exploring the environment, learning from its own experiences, and iteratively self-updating its context about the environment, thereby achieving progressively better performance on future tasks conditioned on the updated context. Major components of the CoD framework include: (1) algorithm design and infrastructure for end-to-end reinforcement learning (RL) with long rollout sequences interleaving solve-task and update-context episodes; (2) tasks and environments for incentivizing and eliciting the targeted meta-capability in LLMs during training, as well as for faithfully measuring progress during evaluation. We present proof-of-concept implementations of the CoD framework, including a GRPO-style RL algorithm with fine-grained credit assignment, as well as tasks and environments tailored to the targeted meta-capability (rather than domain-specific LLM capabilities or standard task-by-task RL). Empirical results validate the efficacy of end-to-end RL training in the CoD setting, and demonstrate the potential for out-of-distribution generalization -- within the training domains, across different domains, and from CoD to Ralph-loop settings -- of the elicited meta-capability. Our investigation of CoD connects several lines of prior works, and opens up new opportunities for advancing LLMs and AI agents. To facilitate further research and applications, we release our implementations at https://github.com/agentscope-ai/Trinity-RFT/tree/research/cod/examples/research_cod.","upvotes":7,"discussionId":"6a39f3f0fdcd3514343bb508","ai_summary":"Large language models can be trained through reinforcement learning to develop a meta-capability enabling continuous learning and adaptation across long sequences of tasks in dynamic environments.","ai_keywords":["large language models","reinforcement learning","long rollout sequences","solve-task episodes","update-context episodes","end-to-end reinforcement learning","GRPO-style RL algorithm","fine-grained credit assignment","meta-capability","long-lifecycle agents","contextual adaptation","out-of-distribution generalization","Ralph-loop settings"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","organization":{"_id":"67d15cca6e2cf0e062dbfb54","name":"AlibabaTongyiLab","fullname":"TongyiLab","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/67d1502bfabfe9974d1f77bb/XdUSVf6HqBzE7zFBfSDQP.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6576f9f4654561a1b345610b","avatarUrl":"/avatars/f801f551640caa70368fcc26a0f51d27.svg","isPro":false,"fullname":"Yanxi Chen","user":"yanxi-chen","type":"user"},{"_id":"6a2da6c8ca070ee12c6e396c","avatarUrl":"/avatars/0355287dcabaa67dbc7f0b10b87451f9.svg","isPro":false,"fullname":"Joe Mama","user":"JoeMama123123123","type":"user"},{"_id":"676c04f44464f476aaa53d1c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/k488J1893F3JGwEMvaeuh.png","isPro":false,"fullname":"Chong Xia","user":"xiac24","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"697c8b15a7f796854ef333c4","avatarUrl":"/avatars/94de3a736fac914944f1b57609e3819a.svg","isPro":false,"fullname":"Joel Wang","user":"joelhenwang","type":"user"},{"_id":"65f3d7ebc2d214f88485bc7d","avatarUrl":"/avatars/d5724567e69e39ec557045a2da237bdd.svg","isPro":false,"fullname":"RagMaster","user":"ragmaster1","type":"user"},{"_id":"69a5cba5ee290d6bb49457b8","avatarUrl":"/avatars/f80c17c13d6baf6bcd375d31efe21116.svg","isPro":true,"fullname":"Darrow O'Lykos","user":"darrowoflykos","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"67d15cca6e2cf0e062dbfb54","name":"AlibabaTongyiLab","fullname":"TongyiLab","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/67d1502bfabfe9974d1f77bb/XdUSVf6HqBzE7zFBfSDQP.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.20002.md","query":{}}">
Connect the Dots: Training LLMs for Long-Lifecycle Agents with Cross-Domain Generalization Via Reinforcement Learning
Abstract
Large language models can be trained through reinforcement learning to develop a meta-capability enabling continuous learning and adaptation across long sequences of tasks in dynamic environments.
This work presents a general framework for training large language models (LLMs) to "Connect the Dots" (CoD), a meta-capability required by long-lifecycle agents: as an LLM-based AI agent gets deployed in an environment, it solves a long sequence of tasks while continuously exploring the environment, learning from its own experiences, and iteratively self-updating its context about the environment, thereby achieving progressively better performance on future tasks conditioned on the updated context. Major components of the CoD framework include: (1) algorithm design and infrastructure for end-to-end reinforcement learning (RL) with long rollout sequences interleaving solve-task and update-context episodes; (2) tasks and environments for incentivizing and eliciting the targeted meta-capability in LLMs during training, as well as for faithfully measuring progress during evaluation. We present proof-of-concept implementations of the CoD framework, including a GRPO-style RL algorithm with fine-grained credit assignment, as well as tasks and environments tailored to the targeted meta-capability (rather than domain-specific LLM capabilities or standard task-by-task RL). Empirical results validate the efficacy of end-to-end RL training in the CoD setting, and demonstrate the potential for out-of-distribution generalization -- within the training domains, across different domains, and from CoD to Ralph-loop settings -- of the elicited meta-capability. Our investigation of CoD connects several lines of prior works, and opens up new opportunities for advancing LLMs and AI agents. To facilitate further research and applications, we release our implementations at https://github.com/agentscope-ai/Trinity-RFT/tree/research/cod/examples/research_cod.
Community
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2606.20002 in a model README.md to link it from this page.
Cite arxiv.org/abs/2606.20002 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2606.20002 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.