Hugging Face Daily Papers · · 4 min read

KVAE: Family of Tokenizers for Multimodal Generative Models

Mirrored from Hugging Face Daily Papers for archival readability. Support the source by reading on the original site.

Report provides results for novel audio, video and image tokenizers. They were developed specifically for subsequent generation by LDM in text-conditioned setting. Series of experiments confirm their ability to surpass leading opensource methods. Inference code available at: <a href=\"https://github.com/kandinskylab/kvae\" rel=\"nofollow\">https://github.com/kandinskylab/kvae</a> with corresponding weights at:</p>\n<ul>\n<li>Audio -- <a href=\"https://huggingface.co/kandinskylab/KVAE-Audio\">https://huggingface.co/kandinskylab/KVAE-Audio</a></li>\n<li>Video<ul>\n<li>4x16x16 -- <a href=\"https://huggingface.co/kandinskylab/KVAE-3D-2.0-t4s16\">https://huggingface.co/kandinskylab/KVAE-3D-2.0-t4s16</a></li>\n<li>4x8x8 -- <a href=\"https://huggingface.co/kandinskylab/KVAE-3D-2.0-t4s8\">https://huggingface.co/kandinskylab/KVAE-3D-2.0-t4s8</a></li>\n</ul>\n</li>\n<li>Image -- <a href=\"https://huggingface.co/kandinskylab/KVAE-2D-2.0\">https://huggingface.co/kandinskylab/KVAE-2D-2.0</a></li>\n</ul>\n","updatedAt":"2026-08-07T18:30:17.354Z","author":{"_id":"68cc16dc1b7328ba01df2e45","avatarUrl":"/avatars/5ab0c5027b33a0680e03d3f4c92ebad3.svg","fullname":"Kirill Chernyshev","name":"rtyasdf","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7709617614746094},"editors":["rtyasdf"],"editorAvatarUrls":["/avatars/5ab0c5027b33a0680e03d3f4c92ebad3.svg"],"reactions":[],"isReport":false},"replies":[{"id":"6a7637007e4e34d640d41810","author":{"_id":"63177d85f957903db971a173","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1665094764329-63177d85f957903db971a173.png","fullname":"Artem","name":"kabachuha","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":78,"isUserFollowing":false},"createdAt":"2026-08-07T19:50:24.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Minimax-destroying Kandinsky-video 6.0 when?1!!1 /s\n\nAwesome work, hope you accomplish what you started and show that we also have something to show the world, not only the West or the East","html":"<p>Minimax-destroying Kandinsky-video 6.0 when?1!!1 /s</p>\n<p>Awesome work, hope you accomplish what you started and show that we also have something to show the world, not only the West or the East</p>\n","updatedAt":"2026-08-07T19:50:24.236Z","author":{"_id":"63177d85f957903db971a173","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1665094764329-63177d85f957903db971a173.png","fullname":"Artem","name":"kabachuha","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":78,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9158686399459839},"editors":["kabachuha"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1665094764329-63177d85f957903db971a173.png"],"reactions":[],"isReport":false,"parentCommentId":"6a76243974be585b61ef7a7a"}}]}],"primaryEmailConfirmed":false,"paper":{"id":"2608.05798","authors":[{"_id":"6a75b8d7e1228e04b3238475","name":"Andrey Shutkin","hidden":false},{"_id":"6a75b8d7e1228e04b3238476","name":"Denis Parkhomenko","hidden":false},{"_id":"6a75b8d7e1228e04b3238477","name":"Ivan Kirillov","hidden":false},{"_id":"6a75b8d7e1228e04b3238478","user":{"_id":"68cc16dc1b7328ba01df2e45","avatarUrl":"/avatars/5ab0c5027b33a0680e03d3f4c92ebad3.svg","isPro":false,"fullname":"Kirill Chernyshev","user":"rtyasdf","type":"user","name":"rtyasdf"},"name":"Kirill Chernyshev","status":"claimed_verified","statusLastChangedAt":"2026-08-07T16:45:28.207Z","hidden":false},{"_id":"6a75b8d7e1228e04b3238479","name":"Kirill Malakhov","hidden":false},{"_id":"6a75b8d7e1228e04b323847a","name":"Ilia Vasiliev","hidden":false},{"_id":"6a75b8d7e1228e04b323847b","name":"Ilia Trushkin","hidden":false},{"_id":"6a75b8d7e1228e04b323847c","name":"Valeriya Kobenko","hidden":false},{"_id":"6a75b8d7e1228e04b323847d","user":{"_id":"664ca5856970d6ba864e0b6f","avatarUrl":"/avatars/4f8f6b0755d8b1ceb48eda1127444e0d.svg","isPro":false,"fullname":"David Chikovani","user":"davidchick","type":"user","name":"davidchick"},"name":"David Chikovani","status":"claimed_verified","statusLastChangedAt":"2026-08-07T16:45:28.202Z","hidden":false},{"_id":"6a75b8d7e1228e04b323847e","user":{"_id":"662b63e9e58d277c811f9cf9","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/662b63e9e58d277c811f9cf9/EKqOzkWnUQEO9m59-a-7u.jpeg","isPro":false,"fullname":"Alexander Ivanov","user":"ivnvalex","type":"user","name":"ivnvalex"},"name":"Alexander Ivanov","status":"claimed_verified","statusLastChangedAt":"2026-08-07T16:45:28.213Z","hidden":false},{"_id":"6a75b8d7e1228e04b323847f","name":"Azat Saginbaev","hidden":false},{"_id":"6a75b8d7e1228e04b3238480","name":"Egor Silvestrov","hidden":false},{"_id":"6a75b8d7e1228e04b3238481","name":"Ivan Mikheev","hidden":false},{"_id":"6a75b8d7e1228e04b3238482","name":"Konstantin Zakharov","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/68cc16dc1b7328ba01df2e45/k_ok8P8J5yeNbLcv1ixIE.png","https://cdn-uploads.huggingface.co/production/uploads/68cc16dc1b7328ba01df2e45/JV1gK1OZ8w5hT4VNruj_8.png","https://cdn-uploads.huggingface.co/production/uploads/68cc16dc1b7328ba01df2e45/6fofDFnZ4QKdaaJcQCBj4.png","https://cdn-uploads.huggingface.co/production/uploads/68cc16dc1b7328ba01df2e45/RLx9vuqbks5HZZsDuRpJ4.png"],"publishedAt":"2026-08-06T00:00:00.000Z","submittedOnDailyAt":"2026-08-07T00:00:00.000Z","title":"KVAE: Family of Tokenizers for Multimodal Generative Models","submittedOnDailyBy":{"_id":"68cc16dc1b7328ba01df2e45","avatarUrl":"/avatars/5ab0c5027b33a0680e03d3f4c92ebad3.svg","isPro":false,"fullname":"Kirill Chernyshev","user":"rtyasdf","type":"user","name":"rtyasdf"},"summary":"Latent diffusion modeling (LDM), a prominent paradigm, utilizes tokenizers to map input signal to compressed representation. This dependency positions tokenizer as an integral part of generation process itself, since it affects learning speed, quality of synthesized samples and lay foundation for later applications. This report presents series of KVAE tokenizers for audio, image and video, all designed for subsequent text-conditioned generation: KVAE-Audio, a continuous full-band 48 kHz tokenizer with a 50 Hz latent of 64 channels; KVAE-3D -- two causal video tokenizers for 4x16x16 and 4x8x8 compression; KVAE-2D, an image model, compressing input by factor of 8 with 32 channels. We demonstrate that reconstruction (PSNR, LPIPS, PESQ, etc.) and generation results on objective (Frechet Distance, CLIP score, CLAP score, etc.) and subjective (side-by-side evaluation) metrics matches or surpasses frontier opensource tokenizers, such as VAEs from Wan-2.2, HunyuanVideo-1.5, FLUX.2, MovieGen, StableAudio and MMAudio. Considering difficulty of development, we share with community training details, model selection method and ablation on design choices. The code is publicly available at https://github.com/kandinskylab/kvae and https://github.com/kandinskylab/kvae-audio.","upvotes":10,"discussionId":"6a75b8d7e1228e04b3238483","githubRepo":"https://github.com/kandinskylab/kvae","githubRepoAddedBy":"user","githubStars":48,"organization":{"_id":"690f6cebeed4fe1e4fafae4c","name":"kandinskylab","fullname":"Kandinsky Lab","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/60991602f7c9c7bf29603a88/oDAZ2f49nmDJKICzgHglO.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"62653f745f6f2e14d6ae128c","avatarUrl":"/avatars/944b564ab810a5b31fa5e45f63bdf4ee.svg","isPro":false,"fullname":"Ivan Kirillov","user":"funnylittleman","type":"user"},{"_id":"691efbe1623e254dabf255eb","avatarUrl":"/avatars/453aba27bca45ae24ff348694a272685.svg","isPro":false,"fullname":"Valery","user":"svaleryk","type":"user"},{"_id":"68cc16dc1b7328ba01df2e45","avatarUrl":"/avatars/5ab0c5027b33a0680e03d3f4c92ebad3.svg","isPro":false,"fullname":"Kirill Chernyshev","user":"rtyasdf","type":"user"},{"_id":"628a57f7042f83532f40c37d","avatarUrl":"/avatars/4b175b1b6f42259ea47a2d804f1070ec.svg","isPro":false,"fullname":"Konstantin","user":"Nikkisora","type":"user"},{"_id":"662b63e9e58d277c811f9cf9","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/662b63e9e58d277c811f9cf9/EKqOzkWnUQEO9m59-a-7u.jpeg","isPro":false,"fullname":"Alexander Ivanov","user":"ivnvalex","type":"user"},{"_id":"6a1094bddb3c66d898cd8b3e","avatarUrl":"/avatars/5a7f546ee3b10d612fba96bb40c176b5.svg","isPro":false,"fullname":"M","user":"kirill-malakhov","type":"user"},{"_id":"664ca5856970d6ba864e0b6f","avatarUrl":"/avatars/4f8f6b0755d8b1ceb48eda1127444e0d.svg","isPro":false,"fullname":"David Chikovani","user":"davidchick","type":"user"},{"_id":"63177d85f957903db971a173","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1665094764329-63177d85f957903db971a173.png","isPro":false,"fullname":"Artem","user":"kabachuha","type":"user"},{"_id":"6687f9a71309e08b1f84bdc6","avatarUrl":"/avatars/f947ec9fe620ae4cffa83b371acdd571.svg","isPro":false,"fullname":"MeiYi","user":"natalie5","type":"user"},{"_id":"66872630abca3bbcec480c94","avatarUrl":"/avatars/b1e4ce1808fddaeb9975f7ba298c86f0.svg","isPro":false,"fullname":"Cris","user":"Bubmy","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"690f6cebeed4fe1e4fafae4c","name":"kandinskylab","fullname":"Kandinsky Lab","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/60991602f7c9c7bf29603a88/oDAZ2f49nmDJKICzgHglO.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.05798.md","query":{}}">
Papers
arxiv:2608.05798

KVAE: Family of Tokenizers for Multimodal Generative Models

Published on Aug 6
· Submitted by
Kirill Chernyshev
on Aug 7

Abstract

Latent diffusion modeling (LDM), a prominent paradigm, utilizes tokenizers to map input signal to compressed representation. This dependency positions tokenizer as an integral part of generation process itself, since it affects learning speed, quality of synthesized samples and lay foundation for later applications. This report presents series of KVAE tokenizers for audio, image and video, all designed for subsequent text-conditioned generation: KVAE-Audio, a continuous full-band 48 kHz tokenizer with a 50 Hz latent of 64 channels; KVAE-3D -- two causal video tokenizers for 4x16x16 and 4x8x8 compression; KVAE-2D, an image model, compressing input by factor of 8 with 32 channels. We demonstrate that reconstruction (PSNR, LPIPS, PESQ, etc.) and generation results on objective (Frechet Distance, CLIP score, CLAP score, etc.) and subjective (side-by-side evaluation) metrics matches or surpasses frontier opensource tokenizers, such as VAEs from Wan-2.2, HunyuanVideo-1.5, FLUX.2, MovieGen, StableAudio and MMAudio. Considering difficulty of development, we share with community training details, model selection method and ablation on design choices. The code is publicly available at https://github.com/kandinskylab/kvae and https://github.com/kandinskylab/kvae-audio.

Community

Paper author Paper submitter about 3 hours ago

Report provides results for novel audio, video and image tokenizers. They were developed specifically for subsequent generation by LDM in text-conditioned setting. Series of experiments confirm their ability to surpass leading opensource methods. Inference code available at: https://github.com/kandinskylab/kvae with corresponding weights at:

Minimax-destroying Kandinsky-video 6.0 when?1!!1 /s

Awesome work, hope you accomplish what you started and show that we also have something to show the world, not only the West or the East

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images

· Sign up or log in to comment

Upvote
10

Get this paper in your agent:

hf papers read 2608.05798
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper

Datasets citing this paper

No dataset linking this paper

Cite arxiv.org/abs/2608.05798 in a dataset README.md to link it from this page.

Spaces citing this paper

No Space linking this paper

Cite arxiv.org/abs/2608.05798 in a Space README.md to link it from this page.

Collections including this paper

No Collection including this paper

Add this paper to a collection to link it from this page.

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from Hugging Face Daily Papers