Sparse Mixture-of-Experts (MoE) layers route tokens through a handful of experts, and learning-free compression of these layers reduces inference cost without retraining. A subtle obstruction blocks every existing compressor in this family: three experts can each be pairwise compatible yet form an irreducible cycle when merged together, so any score that ranks experts on pairwise signals is structurally blind to which triples are jointly mergeable. We show the obstruction is a precise mathematical object, the harmonic kernel of the simplicial Laplacian on a 2-complex whose vertices are experts, whose edges carry KL merge barriers, and whose faces carry triplet barriers; Hodge-decomposing the edge-barrier signal isolates the kernel exactly. We turn the diagnostic into a selection objective: HodgeCover greedily covers the harmonic-critical edges and triplet-critical triangles, and a hybrid variant of HodgeCover pairs it with off-the-shelf weight pruning on survivors. On three open-weight Sparse MoE backbones under aggressive expert reduction, HodgeCover matches state-of-the-art learning-free baselines on the expert-reduction axis, leads on the aggressive-compression frontier of the hybrid axis, and uniquely balances retained mass across all four Hodge components. These results show that exposing the harmonic kernel of a learned MoE structure changes which compressor wins at the regime that matters most.</p>\n","updatedAt":"2026-05-18T01:53:21.201Z","author":{"_id":"69b322027c89a6c6ac5e2474","avatarUrl":"/avatars/811a981d2c59e28f67c3afe6b5c2961e.svg","fullname":"Tao Zhong","name":"n3il666","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8749887347221375},"editors":["n3il666"],"editorAvatarUrls":["/avatars/811a981d2c59e28f67c3afe6b5c2961e.svg"],"reactions":[],"isReport":false}},{"id":"6a0bc14c854731fb631d0c57","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":357,"isUserFollowing":false},"createdAt":"2026-05-19T01:47:56.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [Model Compression with Exact Budget Constraints via Riemannian Manifolds](https://huggingface.co/papers/2605.00649) (2026)\n* [Topological Neural Tangent Kernel](https://huggingface.co/papers/2605.01110) (2026)\n* [TopoU-Net: a U-Net architecture for topological domains](https://huggingface.co/papers/2605.10091) (2026)\n* [Routers Learn the Geometry of Their Experts: Geometric Coupling in Sparse Mixture-of-Experts](https://huggingface.co/papers/2605.12476) (2026)\n* [TopoGeoScore: A Self-Supervised Source-Only Geometric Framework for OOD Checkpoint Selection](https://huggingface.co/papers/2605.08870) (2026)\n* [MoBiE: Efficient Inference of Mixture of Binary Experts under Post-Training Quantization](https://huggingface.co/papers/2604.06798) (2026)\n* [Equifinality in Mixture of Experts: Routing Topology Does Not Determine Language Modeling Quality](https://huggingface.co/papers/2604.14419) (2026)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"<p>This is an automated message from the <a href=\"https://huggingface.co/librarian-bots\">Librarian Bot</a>. I found the following papers similar to this paper. </p>\n<p>The following papers were recommended by the Semantic Scholar API </p>\n<ul>\n<li><a href=\"https://huggingface.co/papers/2605.00649\">Model Compression with Exact Budget Constraints via Riemannian Manifolds</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.01110\">Topological Neural Tangent Kernel</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.10091\">TopoU-Net: a U-Net architecture for topological domains</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.12476\">Routers Learn the Geometry of Their Experts: Geometric Coupling in Sparse Mixture-of-Experts</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.08870\">TopoGeoScore: A Self-Supervised Source-Only Geometric Framework for OOD Checkpoint Selection</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2604.06798\">MoBiE: Efficient Inference of Mixture of Binary Experts under Post-Training Quantization</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2604.14419\">Equifinality in Mixture of Experts: Routing Topology Does Not Determine Language Modeling Quality</a> (2026)</li>\n</ul>\n<p> Please give a thumbs up to this comment if you found it helpful!</p>\n<p> If you want recommendations for any Paper on Hugging Face checkout <a href=\"https://huggingface.co/spaces/librarian-bots/recommend_similar_papers\">this</a> Space</p>\n<p> You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: <code><span class=\"SVELTE_PARTIAL_HYDRATER contents\" data-target=\"UserMention\" data-props=\"{"user":"librarian-bot"}\"><span class=\"inline-block\"><span class=\"contents\"><a href=\"/librarian-bot\">@<span class=\"underline\">librarian-bot</span></a></span> </span></span> recommend</code></p>\n","updatedAt":"2026-05-19T01:47:56.009Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":357,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7388119697570801},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2605.13997","authors":[{"_id":"6a0a708f75184a0d71e02598","user":{"_id":"69b322027c89a6c6ac5e2474","avatarUrl":"/avatars/811a981d2c59e28f67c3afe6b5c2961e.svg","isPro":false,"fullname":"Tao Zhong","user":"n3il666","type":"user","name":"n3il666"},"name":"Tao Zhong","status":"claimed_verified","statusLastChangedAt":"2026-05-18T09:40:52.569Z","hidden":false},{"_id":"6a0a708f75184a0d71e02599","name":"Dongzhe Zheng","hidden":false},{"_id":"6a0a708f75184a0d71e0259a","name":"Christine Allen-Blanchette","hidden":false}],"publishedAt":"2026-05-13T00:00:00.000Z","submittedOnDailyAt":"2026-05-18T00:00:00.000Z","title":"HodgeCover: Higher-Order Topological Coverage Drives Compression of Sparse Mixture-of-Experts","submittedOnDailyBy":{"_id":"69b322027c89a6c6ac5e2474","avatarUrl":"/avatars/811a981d2c59e28f67c3afe6b5c2961e.svg","isPro":false,"fullname":"Tao Zhong","user":"n3il666","type":"user","name":"n3il666"},"summary":"Sparse Mixture-of-Experts (MoE) layers route tokens through a handful of experts, and learning-free compression of these layers reduces inference cost without retraining. A subtle obstruction blocks every existing compressor in this family: three experts can each be pairwise compatible yet form an irreducible cycle when merged together, so any score that ranks experts on pairwise signals is structurally blind to which triples are jointly mergeable. We show the obstruction is a precise mathematical object, the harmonic kernel of the simplicial Laplacian on a 2-complex whose vertices are experts, whose edges carry KL merge barriers, and whose faces carry triplet barriers; Hodge-decomposing the edge-barrier signal isolates the kernel exactly. We turn the diagnostic into a selection objective: HodgeCover greedily covers the harmonic-critical edges and triplet-critical triangles, and a hybrid variant of HodgeCover pairs it with off-the-shelf weight pruning on survivors. On three open-weight Sparse MoE backbones under aggressive expert reduction, HodgeCover matches state-of-the-art learning-free baselines on the expert-reduction axis, leads on the aggressive-compression frontier of the hybrid axis, and uniquely balances retained mass across all four Hodge components. These results show that exposing the harmonic kernel of a learned MoE structure changes which compressor wins at the regime that matters most.","upvotes":3,"discussionId":"6a0a708f75184a0d71e0259b","ai_summary":"A novel compression approach for sparse mixture-of-experts layers uses harmonic kernel analysis from simplicial topology to identify optimal expert merging patterns, enabling efficient inference without retraining.","ai_keywords":["Mixture-of-Experts","sparse MoE","expert routing","learning-free compression","simplicial Laplacian","harmonic kernel","Hodge decomposition","KL divergence","triplet barriers","HodgeCover","weight pruning","expert reduction","aggressive compression"],"organization":{"_id":"64374111a701a7e744c02b0e","name":"princetonu","fullname":"Princeton University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/68e396f2b5bb631e9b2fac9a/b3xXusq8Zz3ej8Z6fRTSZ.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"69b322027c89a6c6ac5e2474","avatarUrl":"/avatars/811a981d2c59e28f67c3afe6b5c2961e.svg","isPro":false,"fullname":"Tao Zhong","user":"n3il666","type":"user"},{"_id":"674572a99543fbaf3c63f35b","avatarUrl":"/avatars/6c891450c2ceeb7b034556548afc772d.svg","isPro":false,"fullname":"蔡正舟","user":"conctsai","type":"user"},{"_id":"699ea5f7ef8843d6e00bf530","avatarUrl":"/avatars/17e1bab801e654a359199d5c29a6a4e5.svg","isPro":false,"fullname":"Benjamin Ocb","user":"benjaminocb","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"64374111a701a7e744c02b0e","name":"princetonu","fullname":"Princeton University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/68e396f2b5bb631e9b2fac9a/b3xXusq8Zz3ej8Z6fRTSZ.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2605/2605.13997.md"}">
HodgeCover: Higher-Order Topological Coverage Drives Compression of Sparse Mixture-of-Experts
Abstract
A novel compression approach for sparse mixture-of-experts layers uses harmonic kernel analysis from simplicial topology to identify optimal expert merging patterns, enabling efficient inference without retraining.
AI-generated summary
Sparse Mixture-of-Experts (MoE) layers route tokens through a handful of experts, and learning-free compression of these layers reduces inference cost without retraining. A subtle obstruction blocks every existing compressor in this family: three experts can each be pairwise compatible yet form an irreducible cycle when merged together, so any score that ranks experts on pairwise signals is structurally blind to which triples are jointly mergeable. We show the obstruction is a precise mathematical object, the harmonic kernel of the simplicial Laplacian on a 2-complex whose vertices are experts, whose edges carry KL merge barriers, and whose faces carry triplet barriers; Hodge-decomposing the edge-barrier signal isolates the kernel exactly. We turn the diagnostic into a selection objective: HodgeCover greedily covers the harmonic-critical edges and triplet-critical triangles, and a hybrid variant of HodgeCover pairs it with off-the-shelf weight pruning on survivors. On three open-weight Sparse MoE backbones under aggressive expert reduction, HodgeCover matches state-of-the-art learning-free baselines on the expert-reduction axis, leads on the aggressive-compression frontier of the hybrid axis, and uniquely balances retained mass across all four Hodge components. These results show that exposing the harmonic kernel of a learned MoE structure changes which compressor wins at the regime that matters most.
Community
Sparse Mixture-of-Experts (MoE) layers route tokens through a handful of experts, and learning-free compression of these layers reduces inference cost without retraining. A subtle obstruction blocks every existing compressor in this family: three experts can each be pairwise compatible yet form an irreducible cycle when merged together, so any score that ranks experts on pairwise signals is structurally blind to which triples are jointly mergeable. We show the obstruction is a precise mathematical object, the harmonic kernel of the simplicial Laplacian on a 2-complex whose vertices are experts, whose edges carry KL merge barriers, and whose faces carry triplet barriers; Hodge-decomposing the edge-barrier signal isolates the kernel exactly. We turn the diagnostic into a selection objective: HodgeCover greedily covers the harmonic-critical edges and triplet-critical triangles, and a hybrid variant of HodgeCover pairs it with off-the-shelf weight pruning on survivors. On three open-weight Sparse MoE backbones under aggressive expert reduction, HodgeCover matches state-of-the-art learning-free baselines on the expert-reduction axis, leads on the aggressive-compression frontier of the hybrid axis, and uniquely balances retained mass across all four Hodge components. These results show that exposing the harmonic kernel of a learned MoE structure changes which compressor wins at the regime that matters most.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2605.13997 in a model README.md to link it from this page.
Cite arxiv.org/abs/2605.13997 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2605.13997 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.