We present LongStraw, an architecture-aware execution stack for million-token RL post-training under a fixed GPU budget, instantiated with Group Relative Policy Optimization (GRPO). LongStraw evaluates the shared prompt once without automatic differentiation, retains only the model-specific state required by later tokens, and replays short response branches one at a time under autograd. This reduces the live training graph from the full prompt and response sequence to a single response branch, trading additional replay time for lower GPU memory usage. We implement LongStraw for two substantially different model families: the hybrid recurrent and full-attention Qwen3.6-27B, and the compressed-attention mixture-of-experts GLM-5.2.</p>\n","updatedAt":"2026-07-17T09:46:05.191Z","author":{"_id":"677aa4d51d4f6bcb2e217c93","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/BPjftZ9rVhaGJfEBRDmLb.png","fullname":"Xiaoteng Ma","name":"xtma","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":3,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8808848857879639},"editors":["xtma"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/BPjftZ9rVhaGJfEBRDmLb.png"],"reactions":[],"isReport":false}},{"id":"6a5a1be8b7165b257fe90bc9","author":{"_id":"658412f93a84a40185adaf37","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/658412f93a84a40185adaf37/FKXH7e1jj09KO1v-B5sER.jpeg","fullname":"Aamer Mihaysi","name":"O96a","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false},"createdAt":"2026-07-17T12:11:20.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"I'd love to see how this holds up when the agent trajectories aren't cleanly segmented into prompt + generation. Most agent loops I run have tool outputs streaming back mid-trajectory, which means the \"shared prompt\" assumption gets fuzzy fast — the model sees partial context, then more context, then a tool result that rewrites the whole thing. The fixed-GPU-budget angle is the real draw here, but I suspect the eval needs a version where context grows unpredictably, not just long. If the architecture handles that, it's genuinely useful for anyone running agents on a cluster they don't own.","html":"<p>I'd love to see how this holds up when the agent trajectories aren't cleanly segmented into prompt + generation. Most agent loops I run have tool outputs streaming back mid-trajectory, which means the \"shared prompt\" assumption gets fuzzy fast — the model sees partial context, then more context, then a tool result that rewrites the whole thing. The fixed-GPU-budget angle is the real draw here, but I suspect the eval needs a version where context grows unpredictably, not just long. If the architecture handles that, it's genuinely useful for anyone running agents on a cluster they don't own.</p>\n","updatedAt":"2026-07-17T12:11:20.521Z","author":{"_id":"658412f93a84a40185adaf37","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/658412f93a84a40185adaf37/FKXH7e1jj09KO1v-B5sER.jpeg","fullname":"Aamer Mihaysi","name":"O96a","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9056068062782288},"editors":["O96a"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/658412f93a84a40185adaf37/FKXH7e1jj09KO1v-B5sER.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2607.14952","authors":[{"_id":"6a59f984d85523d2cb93cf31","name":"Changhai Zhou","hidden":false},{"_id":"6a59f984d85523d2cb93cf32","name":"Kieran Liu","hidden":false},{"_id":"6a59f984d85523d2cb93cf33","name":"Yuhua Zhou","hidden":false},{"_id":"6a59f984d85523d2cb93cf34","name":"Qian Qiao","hidden":false},{"_id":"6a59f984d85523d2cb93cf35","name":"Jun Gao","hidden":false},{"_id":"6a59f984d85523d2cb93cf36","name":"Harry Zhang","hidden":false},{"_id":"6a59f984d85523d2cb93cf37","name":"Irvine Lu","hidden":false},{"_id":"6a59f984d85523d2cb93cf38","name":"Nolan Ho","hidden":false},{"_id":"6a59f984d85523d2cb93cf39","name":"Lucian Li","hidden":false},{"_id":"6a59f984d85523d2cb93cf3a","name":"Andrew Lei","hidden":false},{"_id":"6a59f984d85523d2cb93cf3b","name":"Cleon Cheng","hidden":false},{"_id":"6a59f984d85523d2cb93cf3c","name":"Steven Chiang","hidden":false},{"_id":"6a59f984d85523d2cb93cf3d","name":"Yihang Zeng","hidden":false},{"_id":"6a59f984d85523d2cb93cf3e","name":"Di Zhang","hidden":false},{"_id":"6a59f984d85523d2cb93cf3f","name":"Rio Yang","hidden":false},{"_id":"6a59f984d85523d2cb93cf40","name":"Kaijie Chen","hidden":false},{"_id":"6a59f984d85523d2cb93cf41","name":"Andrew Chen","hidden":false},{"_id":"6a59f984d85523d2cb93cf42","name":"Pony Ma","hidden":false},{"_id":"6a59f984d85523d2cb93cf43","name":"Weizhong Zhang","hidden":false},{"_id":"6a59f984d85523d2cb93cf44","name":"Cheng Jin","hidden":false}],"publishedAt":"2026-07-16T00:00:00.000Z","submittedOnDailyAt":"2026-07-17T00:00:00.000Z","title":"LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget","submittedOnDailyBy":{"_id":"677aa4d51d4f6bcb2e217c93","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/BPjftZ9rVhaGJfEBRDmLb.png","isPro":false,"fullname":"Xiaoteng Ma","user":"xtma","type":"user","name":"xtma"},"summary":"A growing gap separates inference context lengths from RL post-training: inference systems are approaching million-token contexts, while post-training workloads often remain at 256K tokens or below and rely on length generalization at deployment. The gap is especially important for AI agents, whose observations, tool outputs, documents, and prior decisions accumulate over long trajectories. LongStraw is an architecture-aware execution stack for million-token RL post-training under a fixed GPU budget, instantiated with Group Relative Policy Optimization (GRPO). It evaluates the shared prompt without autograd, retains only model-specific state needed by later tokens, and replays short response branches one at a time, reducing the live training graph at the cost of additional replay time. We implement it for the hybrid recurrent and full-attention Qwen3.6-27B and the compressed-attention mixture-of-experts GLM-5.2. On eight H20 GPUs, LongStraw completes grouped Qwen scoring and response backward at 2.1M positions for groups of 2 and 8; increasing the group size adds only 0.21 GB of peak allocated memory, while a separate stress test reaches 4.46M positions. On 32 H20 GPUs, we validate the end-to-end LongStraw execution path for a 2.1M-token prompt across all 78 layers of GLM-5.2. These experiments establish execution capacity rather than complete training correctness because the captured prompt state is detached and some distributed forward and gradient composition paths remain incomplete.","upvotes":33,"discussionId":"6a59f984d85523d2cb93cf45","githubRepo":"https://github.com/MindLab-Research/longstraw","githubRepoAddedBy":"user","githubStars":0,"organization":{"_id":"69d05e49e99e437a3b18bebc","name":"mindlab-research","fullname":"Mind Lab","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/64303a4484f3ed1ce62a2c5a/0L3sMuvL2JGls3zUKOKYU.jpeg"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"68418e5b5de370509569530d","avatarUrl":"/avatars/2393d8ea9fd21d9f27156c29b95aa0a1.svg","isPro":false,"fullname":"Ruijian Ye","user":"biubiu0002","type":"user"},{"_id":"6969a15c87ebb192324c37e8","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/qWv0dF3x7ZtKdn0MqrjXW.png","isPro":false,"fullname":"Qihan Liu","user":"liuqh16","type":"user"},{"_id":"677aa4d51d4f6bcb2e217c93","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/BPjftZ9rVhaGJfEBRDmLb.png","isPro":false,"fullname":"Xiaoteng Ma","user":"xtma","type":"user"},{"_id":"668775b5c82095d2543b47a3","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/gPgYQAj_WOqTtehiWIa-6.jpeg","isPro":false,"fullname":"NolanHo","user":"NolanHo","type":"user"},{"_id":"64a0154fbfd4d3ffab930448","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64a0154fbfd4d3ffab930448/lv3tcTlDO-GkjN3kJx5RQ.jpeg","isPro":false,"fullname":"Hongmutian","user":"OriReplication","type":"user"},{"_id":"6a05ddeaadf2f32ebb53e9b4","avatarUrl":"/avatars/61362856e551fd32cb69210d020c1360.svg","isPro":false,"fullname":"hua","user":"johnsmithhua77","type":"user"},{"_id":"65d4a5788fa4ad07bc29a48b","avatarUrl":"/avatars/ee70eaae8e19ab0472ff72eb3992fb2f.svg","isPro":false,"fullname":"y","user":"rioy","type":"user"},{"_id":"6a03e53624974fbafc902240","avatarUrl":"/avatars/1902dbabc9c8ad05e532050cae69c545.svg","isPro":false,"fullname":"Hua","user":"peixuan77","type":"user"},{"_id":"6621e02cf34ab6caed18e9c6","avatarUrl":"/avatars/15888b2060d1cc56be9fa55fd4b34005.svg","isPro":false,"fullname":"Fanqi Kong","user":"Fancylalala","type":"user"},{"_id":"6621e57973c423da59461b65","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6621e57973c423da59461b65/dbbMP6c7rGA7um97_OKGY.jpeg","isPro":false,"fullname":"Menghuan","user":"Menghuan1918","type":"user"},{"_id":"69ad58700fecd2cb1528305d","avatarUrl":"/avatars/0f0718faabbceede61891cd9afd6a951.svg","isPro":false,"fullname":"Hookxmum","user":"Hookxmum","type":"user"},{"_id":"6a054e9f7c366f1693daec6e","avatarUrl":"/avatars/0aeafc451dafb40ca3e024c7ecb67445.svg","isPro":false,"fullname":"Kki","user":"Yinjiacheng","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"69d05e49e99e437a3b18bebc","name":"mindlab-research","fullname":"Mind Lab","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/64303a4484f3ed1ce62a2c5a/0L3sMuvL2JGls3zUKOKYU.jpeg"},"query":{}}">
LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget
Abstract
A growing gap separates inference context lengths from RL post-training: inference systems are approaching million-token contexts, while post-training workloads often remain at 256K tokens or below and rely on length generalization at deployment. The gap is especially important for AI agents, whose observations, tool outputs, documents, and prior decisions accumulate over long trajectories. LongStraw is an architecture-aware execution stack for million-token RL post-training under a fixed GPU budget, instantiated with Group Relative Policy Optimization (GRPO). It evaluates the shared prompt without autograd, retains only model-specific state needed by later tokens, and replays short response branches one at a time, reducing the live training graph at the cost of additional replay time. We implement it for the hybrid recurrent and full-attention Qwen3.6-27B and the compressed-attention mixture-of-experts GLM-5.2. On eight H20 GPUs, LongStraw completes grouped Qwen scoring and response backward at 2.1M positions for groups of 2 and 8; increasing the group size adds only 0.21 GB of peak allocated memory, while a separate stress test reaches 4.46M positions. On 32 H20 GPUs, we validate the end-to-end LongStraw execution path for a 2.1M-token prompt across all 78 layers of GLM-5.2. These experiments establish execution capacity rather than complete training correctness because the captured prompt state is detached and some distributed forward and gradient composition paths remain incomplete.
Community
We present LongStraw, an architecture-aware execution stack for million-token RL post-training under a fixed GPU budget, instantiated with Group Relative Policy Optimization (GRPO). LongStraw evaluates the shared prompt once without automatic differentiation, retains only the model-specific state required by later tokens, and replays short response branches one at a time under autograd. This reduces the live training graph from the full prompt and response sequence to a single response branch, trading additional replay time for lower GPU memory usage. We implement LongStraw for two substantially different model families: the hybrid recurrent and full-attention Qwen3.6-27B, and the compressed-attention mixture-of-experts GLM-5.2.
I'd love to see how this holds up when the agent trajectories aren't cleanly segmented into prompt + generation. Most agent loops I run have tool outputs streaming back mid-trajectory, which means the "shared prompt" assumption gets fuzzy fast — the model sees partial context, then more context, then a tool result that rewrites the whole thing. The fixed-GPU-budget angle is the real draw here, but I suspect the eval needs a version where context grows unpredictably, not just long. If the architecture handles that, it's genuinely useful for anyone running agents on a cluster they don't own.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2607.14952 in a model README.md to link it from this page.
Cite arxiv.org/abs/2607.14952 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2607.14952 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.