Two studies define measurable GPU control gates for LLM-agent services by analyzing concurrent cohort scheduling and on-device routing versus host redispatch.</p>\n","updatedAt":"2026-08-13T09:16:30.585Z","author":{"_id":"64442f46af034cdfd69d5bc4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64442f46af034cdfd69d5bc4/xBD3PKu6sOMAKVsvi4hQ6.jpeg","fullname":"Josef Chen","name":"josefchen","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":3,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8800403475761414},"editors":["josefchen"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/64442f46af034cdfd69d5bc4/xBD3PKu6sOMAKVsvi4hQ6.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.12123","authors":[{"_id":"6a7d8b2642823931a1f1739b","name":"Josef Liyanjun Chen","hidden":false}],"publishedAt":"2026-08-12T00:00:00.000Z","submittedOnDailyAt":"2026-08-13T00:00:00.000Z","title":"Ready Cohorts: Bounding GPU Opportunity and Avoiding Host Round Trips in LLM-Agent Control","submittedOnDailyBy":{"_id":"64442f46af034cdfd69d5bc4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64442f46af034cdfd69d5bc4/xBD3PKu6sOMAKVsvi4hQ6.jpeg","isPro":true,"fullname":"Josef Chen","user":"josefchen","type":"user","name":"josefchen"},"summary":"LLM-agent services repeatedly execute small deterministic transitions between model and tool calls: route an outcome, update state, and emit the next effect. We ask when this control path exposes enough concurrent work for GPU execution, and what changes when a GPU-computed route decision remains on device. We formalize the ready-cohort boundary using fixed-partition share F, exact offline share P*, local upper bound U, and online achieved share A. Under zero service time, unlimited capacity, and equal relative launch deadlines, a specialized dynamic program computes P* exactly. In a stationary Poisson replay of one pinned 851-session public trace panel, the primary condition at 100,000 target active sessions, K=256, and a 50 ms launch deadline gives F=30.19%, P*=43.00%, and U=45.85%. Exact packing recovers 81.83% of the opportunity lost at fixed window boundaries. The outcome-derived route key is a conditioning proxy, not proof of executable identity. A separate mechanism study keeps a GPU-computed binary decision on device instead of returning four bytes to the host and redispatching. Across four named GPU placements, the device-resident path is faster in all 36 configurations; within-placement row-median ratios range from 1.19x to 2.39x. Across both admissible mechanisms, all 14,557,440 tested batched invocations match a separately implemented host oracle. A fixed nested device graph that removes no host decision is slower in all 60 configurations across five placements. Together, the studies establish two measurable gates for GPU agent control: deadline-feasible cohort supply and observation placement. A joined finite online runtime is required to measure A, CPU displacement, and service-level benefit.","upvotes":1,"discussionId":"6a7d8b2742823931a1f1739c","projectPage":"https://huggingface.co/spaces/josefchen/ready-cohorts","githubRepo":"https://github.com/josefchen/ready-cohorts","githubRepoAddedBy":"user","ai_summary":"Two studies define measurable GPU control gates for LLM-agent services by analyzing concurrent cohort scheduling and on-device routing versus host redispatch.","ai_keywords":["LLM-agent","GPU execution","dynamic program","Poisson replay","cohort boundary","launch deadline","packing","GPU-computed binary decision","device-resident path","batched invocations","nested device graph","observation placement"],"ai_summary_model":"thinkingmachines/Inkling-Small","githubStars":0},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"64442f46af034cdfd69d5bc4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64442f46af034cdfd69d5bc4/xBD3PKu6sOMAKVsvi4hQ6.jpeg","isPro":true,"fullname":"Josef Chen","user":"josefchen","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.12123.md","query":{}}">
Ready Cohorts: Bounding GPU Opportunity and Avoiding Host Round Trips in LLM-Agent Control
Abstract
Two studies define measurable GPU control gates for LLM-agent services by analyzing concurrent cohort scheduling and on-device routing versus host redispatch.
LLM-agent services repeatedly execute small deterministic transitions between model and tool calls: route an outcome, update state, and emit the next effect. We ask when this control path exposes enough concurrent work for GPU execution, and what changes when a GPU-computed route decision remains on device. We formalize the ready-cohort boundary using fixed-partition share F, exact offline share P*, local upper bound U, and online achieved share A. Under zero service time, unlimited capacity, and equal relative launch deadlines, a specialized dynamic program computes P* exactly. In a stationary Poisson replay of one pinned 851-session public trace panel, the primary condition at 100,000 target active sessions, K=256, and a 50 ms launch deadline gives F=30.19%, P*=43.00%, and U=45.85%. Exact packing recovers 81.83% of the opportunity lost at fixed window boundaries. The outcome-derived route key is a conditioning proxy, not proof of executable identity. A separate mechanism study keeps a GPU-computed binary decision on device instead of returning four bytes to the host and redispatching. Across four named GPU placements, the device-resident path is faster in all 36 configurations; within-placement row-median ratios range from 1.19x to 2.39x. Across both admissible mechanisms, all 14,557,440 tested batched invocations match a separately implemented host oracle. A fixed nested device graph that removes no host decision is slower in all 60 configurations across five placements. Together, the studies establish two measurable gates for GPU agent control: deadline-feasible cohort supply and observation placement. A joined finite online runtime is required to measure A, CPU displacement, and service-level benefit.
Community
Two studies define measurable GPU control gates for LLM-agent services by analyzing concurrent cohort scheduling and on-device routing versus host redispatch.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2608.12123 in a model README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.