TLDR: improvement of TurboQuant by using octahedral parametrization. Shows improvement on all bit depth in various modalities (text, video, audio).</p>\n","updatedAt":"2026-05-21T05:37:03.454Z","author":{"_id":"64b7f06fda8017900e893eb4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64b7f06fda8017900e893eb4/3VcQFjERXyAjHQFLMFEJt.jpeg","fullname":"Mark Boss","name":"mboss","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":36,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7884619832038879},"editors":["mboss"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/64b7f06fda8017900e893eb4/3VcQFjERXyAjHQFLMFEJt.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2605.21226","authors":[{"_id":"6a0e9908164dbbc68a26c648","name":"Mark Boss","hidden":false},{"_id":"6a0e9908164dbbc68a26c649","name":"Vikram Voleti","hidden":false},{"_id":"6a0e9908164dbbc68a26c64a","name":"Simon Donné","hidden":false},{"_id":"6a0e9908164dbbc68a26c64b","name":"Shimon Vainer","hidden":false}],"publishedAt":"2026-05-20T00:00:00.000Z","submittedOnDailyAt":"2026-05-21T00:00:00.000Z","title":"OCTOPUS: Optimized KV Cache for Transformers via Octahedral Parametrization Under optimal Squared error quantization","submittedOnDailyBy":{"_id":"64b7f06fda8017900e893eb4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64b7f06fda8017900e893eb4/3VcQFjERXyAjHQFLMFEJt.jpeg","isPro":false,"fullname":"Mark Boss","user":"mboss","type":"user","name":"mboss"},"summary":"The key-value (KV) cache dominates memory bandwidth and footprint in long-context autoregressive inference. Recent rotation-preconditioned codecs (TurboQuant, PolarQuant) show that a structured random rotation followed by a per-coordinate scalar quantizer matched to an analytically tractable marginal is a near-optimal recipe for KV compression. OCTOPUS advances this paradigm through joint quantization of rotated coordinate triplets. Each triplet's direction is mapped to a square via an octahedral parameterization, and the two resulting coordinates and the triplet norm are Lloyd-Max quantized against implementation-matched marginals. Optimizing the per-triplet squared error gives a strictly non-uniform bit allocation depending only on the total dimensionality of the keys. We find the finite-dimensional quality optimum with sweeps to be constant on every real decoder we test. The codec is data-oblivious, online, and deterministic given a seed. Across text, video, and audio, OCTOPUS matches or beats every prior rotation codec at every reported bit width and metric, with a lead that grows as bits drop for extreme compression. Furthermore, a fused Triton implementation reconstructs keys on the fly without materializing the uncompressed key, so the codec adds no decode-time bandwidth or latency over the existing dequantization. Project Page: https://octopus-quant.github.io/","upvotes":3,"discussionId":"6a0e9908164dbbc68a26c64c","projectPage":"https://octopus-quant.github.io/","ai_summary":"OCTOPUS achieves efficient key-value cache compression through structured random rotations and optimized quantization of coordinate triplets, enabling high-quality reconstruction with reduced memory bandwidth usage.","ai_keywords":["key-value cache","rotation-preconditioned codecs","TurboQuant","PolarQuant","structured random rotation","per-coordinate scalar quantizer","analytical tractable marginal","latent diffusion models","cross-attention layers","image inpainting","unconditional image generation","semantic scene synthesis","super-resolution"],"organization":{"_id":"62e1573a6fb6e362b4a90690","name":"stabilityai","fullname":"Stability AI","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/643feeb67bc3fbde1385cc25/7vmYr2XwVcPtkLzac_jxQ.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"64b7f06fda8017900e893eb4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64b7f06fda8017900e893eb4/3VcQFjERXyAjHQFLMFEJt.jpeg","isPro":false,"fullname":"Mark Boss","user":"mboss","type":"user"},{"_id":"698309b43ea114b8ee0f921e","avatarUrl":"/avatars/d4c942d84879cfb6663f6ea8f4b16995.svg","isPro":false,"fullname":"Phyo Phyo","user":"ugh-45","type":"user"},{"_id":"6997ef2f68950cfdb9f81875","avatarUrl":"/avatars/d99a1f9df211f4162b4e177eded49570.svg","isPro":false,"fullname":"Jcdbzh9olj","user":"jcdbzh9olj","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"62e1573a6fb6e362b4a90690","name":"stabilityai","fullname":"Stability AI","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/643feeb67bc3fbde1385cc25/7vmYr2XwVcPtkLzac_jxQ.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2605/2605.21226.md"}">
OCTOPUS: Optimized KV Cache for Transformers via Octahedral Parametrization Under optimal Squared error quantization
Abstract
OCTOPUS achieves efficient key-value cache compression through structured random rotations and optimized quantization of coordinate triplets, enabling high-quality reconstruction with reduced memory bandwidth usage.
AI-generated summary
The key-value (KV) cache dominates memory bandwidth and footprint in long-context autoregressive inference. Recent rotation-preconditioned codecs (TurboQuant, PolarQuant) show that a structured random rotation followed by a per-coordinate scalar quantizer matched to an analytically tractable marginal is a near-optimal recipe for KV compression. OCTOPUS advances this paradigm through joint quantization of rotated coordinate triplets. Each triplet's direction is mapped to a square via an octahedral parameterization, and the two resulting coordinates and the triplet norm are Lloyd-Max quantized against implementation-matched marginals. Optimizing the per-triplet squared error gives a strictly non-uniform bit allocation depending only on the total dimensionality of the keys. We find the finite-dimensional quality optimum with sweeps to be constant on every real decoder we test. The codec is data-oblivious, online, and deterministic given a seed. Across text, video, and audio, OCTOPUS matches or beats every prior rotation codec at every reported bit width and metric, with a lead that grows as bits drop for extreme compression. Furthermore, a fused Triton implementation reconstructs keys on the fly without materializing the uncompressed key, so the codec adds no decode-time bandwidth or latency over the existing dequantization. Project Page: https://octopus-quant.github.io/
Community
TLDR: improvement of TurboQuant by using octahedral parametrization. Shows improvement on all bit depth in various modalities (text, video, audio).
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2605.21226 in a model README.md to link it from this page.
Cite arxiv.org/abs/2605.21226 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2605.21226 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.