MergePipe is the first parameter management and execution system for large-scale LLM merging.</p>\n<p>It reframes LLM merging as a data management problem, rather than a one-off script execution:<br>explicitly model expert parameter reads as a budgeted resource,<br>explicitly planning merges before execution, and<br>materialize merged checkpoints with atomic publish and immutable manifests.</p>\n","updatedAt":"2026-06-04T03:21:51.728Z","author":{"_id":"67cb0cc7ba670f30e55d46d4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/67cb0cc7ba670f30e55d46d4/veKpvPWzKrdXTnXFIWR1C.jpeg","fullname":"Yuanyi Wang","name":"wyy-code","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":5,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8928650617599487},"editors":["wyy-code"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/67cb0cc7ba670f30e55d46d4/veKpvPWzKrdXTnXFIWR1C.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2605.29489","authors":[{"_id":"6a202b5f15100c5272a84180","name":"Yuanyi Wang","hidden":false},{"_id":"6a202b5f15100c5272a84181","name":"Yanggan Gu","hidden":false},{"_id":"6a202b5f15100c5272a84182","name":"Su Lu","hidden":false},{"_id":"6a202b5f15100c5272a84183","name":"Yifan Yang","hidden":false},{"_id":"6a202b5f15100c5272a84184","name":"Zhaoyi Yan","hidden":false},{"_id":"6a202b5f15100c5272a84185","name":"Congkai Xie","hidden":false},{"_id":"6a202b5f15100c5272a84186","name":"Jianmin Wu","hidden":false},{"_id":"6a202b5f15100c5272a84187","name":"Hongxia Yang","hidden":false}],"publishedAt":"2026-05-28T00:00:00.000Z","submittedOnDailyAt":"2026-06-04T00:00:00.000Z","title":"Access Sets Matter: Budgeting Expert Reads for Scalable Weight-Space Model Merging","submittedOnDailyBy":{"_id":"67cb0cc7ba670f30e55d46d4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/67cb0cc7ba670f30e55d46d4/veKpvPWzKrdXTnXFIWR1C.jpeg","isPro":false,"fullname":"Yuanyi Wang","user":"wyy-code","type":"user","name":"wyy-code"},"summary":"Weight-space model merging is usually formulated as an algebraic operation on checkpoints, yet at LLM scale the limiting resource is often the set of expert weights that must be read. We introduce MergePipe, a budget-aware execution layer that casts LLM merging as an expert access-set problem: given a merge operator and a checkpoint family in a shared weight coordinate system, choose which expert delta blocks to access under an explicit I/O budget. MergePipe indexes parameter blocks, builds deterministic access plans, and executes the induced budgeted merge with replayable manifests. The plan is budget-sound by construction and recovers the full-read merge at full budget; for fixed-coefficient additive operators, the omitted-update error is bounded by the norm of omitted deltas. Across Qwen and Llama merging workloads, MergePipe reduces expert-read I/O by up to an order of magnitude and achieves up to 11times speedups. Representative budget sweeps show O(10^{-3}) parameter deviation from full-read merges and no monotonic degradation on downstream benchmarks.","upvotes":4,"discussionId":"6a202b6015100c5272a84188","githubRepo":"https://github.com/wyy-code/mergepipe","githubRepoAddedBy":"user","ai_summary":"MergePipe addresses expert weight access limitations in large language model merging by formulating it as an expert access-set problem with budget-aware execution and deterministic planning.","ai_keywords":["weight-space model merging","checkpoints","expert weights","I/O budget","merge operator","checkpoint family","shared weight coordinate system","expert delta blocks","budget-aware execution","deterministic access plans","replayable manifests","budget-sound","omitted-update error","parameter blocks"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","githubStars":2,"organization":{"_id":"646ecc368d316fde87b3b6e3","name":"PolyUHK","fullname":"The Hong Kong Polytechnic University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/646ecbc0cbb7bb996513e298/Akb4zKqIP9kb9PQoUPUmj.jpeg"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"67cb0cc7ba670f30e55d46d4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/67cb0cc7ba670f30e55d46d4/veKpvPWzKrdXTnXFIWR1C.jpeg","isPro":false,"fullname":"Yuanyi Wang","user":"wyy-code","type":"user"},{"_id":"643e4ad11d0e956d94ba65d4","avatarUrl":"/avatars/69236b1cdb65031fe425b92dfb12b324.svg","isPro":false,"fullname":"slu","user":"sslu","type":"user"},{"_id":"662bba49bed98acbe616d37d","avatarUrl":"/avatars/f70ded35f371ec0d10249d4248d3cea1.svg","isPro":false,"fullname":"yanggangu","user":"yanggangu","type":"user"},{"_id":"69cccf1d4cf51250c2a061b5","avatarUrl":"/avatars/604b7003800a6b332fc97fbc196d5ecb.svg","isPro":false,"fullname":"Sofia Johnson","user":"theodorescott69","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"646ecc368d316fde87b3b6e3","name":"PolyUHK","fullname":"The Hong Kong Polytechnic University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/646ecbc0cbb7bb996513e298/Akb4zKqIP9kb9PQoUPUmj.jpeg"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2605/2605.29489.md"}">

Get this paper in your agent:

Don't have the latest CLI?

Cite arxiv.org/abs/2605.29489 in a model README.md to link it from this page.

Cite arxiv.org/abs/2605.29489 in a dataset README.md to link it from this page.

Cite arxiv.org/abs/2605.29489 in a Space README.md to link it from this page.

Add this paper to a collection to link it from this page.