Paper: <a href=\"https://arxiv.org/abs/2608.00079\" rel=\"nofollow\">https://arxiv.org/abs/2608.00079</a><br>Project Page: <a href=\"https://zhangrongxiang.github.io/leaptalk-page/\" rel=\"nofollow\">https://zhangrongxiang.github.io/leaptalk-page/</a><br>Code: <a href=\"https://github.com/zhangrongxiang/LeapTalk\" rel=\"nofollow\">https://github.com/zhangrongxiang/LeapTalk</a></p>\n","updatedAt":"2026-08-04T04:58:34.510Z","author":{"_id":"65f30652b0d359b2ffa4a42c","avatarUrl":"/avatars/f6cb0705b25cc0ccc0b099811ef9a871.svg","fullname":"zrx","name":"z-rx","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.41854003071784973},"editors":["z-rx"],"editorAvatarUrls":["/avatars/f6cb0705b25cc0ccc0b099811ef9a871.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.00079","authors":[{"_id":"6a7157f7ec5082b9f872ce15","name":"Rongxiang Zhang","hidden":false},{"_id":"6a7157f7ec5082b9f872ce16","name":"Songhua Liu","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/65f30652b0d359b2ffa4a42c/nCr8rc-3B4R8Pl5vyqJh9.mp4"],"publishedAt":"2026-07-29T00:00:00.000Z","submittedOnDailyAt":"2026-08-04T00:00:00.000Z","title":"LeapTalk: Breaking the Latency-Quality Trade-off in Talking Head Generation","submittedOnDailyBy":{"_id":"65f30652b0d359b2ffa4a42c","avatarUrl":"/avatars/f6cb0705b25cc0ccc0b099811ef9a871.svg","isPro":false,"fullname":"zrx","user":"z-rx","type":"user","name":"z-rx"},"summary":"Long-form and real-time talking-head generation remains challenging due to a latency-quality trade-off: inefficient multi-step diffusion prohibits streaming generation, whereas real-time autoregressive approaches suffer from error accumulation and identity drift. To address this drawback, we propose LeapTalk, a novel framework that achieves stable and real-time talking-head generation with a single forward step, scaling to arbitrarily long videos. At the heart of our approach lies a single-step bridge distillation scheme. On the one hand, departing from the conventional noise-to-data paradigm, we introduce a data-to-data transport formulation based on a Brownian bridge. Anchored by a persistent reference, this strategy effectively mitigates identity drift and enhances long-term temporal stability. On the other hand, to enable smooth knowledge transfer from a pre-trained diffusion teacher to the student bridge model, we explore a heterogeneous distillation framework with an SNR-aligned time transformation Φ(τ), which bridges the functional discrepancy between the two models. Moreover, we propose an audio-driven classifier-free guidance mechanism to maintain fine-grained lip synchronization under extreme step reduction. Extensive experiments demonstrate that our method achieves high-fidelity and temporally consistent video generation with only 1 step at up to 200 FPS, significantly outperforming existing approaches in both efficiency and stability. Project Page: https://zhangrongxiang.github.io/leaptalk-page/","upvotes":4,"discussionId":"6a7157f7ec5082b9f872ce17","projectPage":"https://zhangrongxiang.github.io/leaptalk-page/","githubRepo":"https://github.com/zhangrongxiang/LeapTalk","githubRepoAddedBy":"user","githubStars":3,"organization":{"_id":"686ccb41f10ff747aaea3f13","name":"Shanghai-Jiao-Tong-University-SAI","fullname":"Shanghai Jiao Tong University SAI","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/686cc925516900c6865a1d56/KmvImL_J2ZEHdVFumR3pO.jpeg"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"65f30652b0d359b2ffa4a42c","avatarUrl":"/avatars/f6cb0705b25cc0ccc0b099811ef9a871.svg","isPro":false,"fullname":"zrx","user":"z-rx","type":"user"},{"_id":"64b929308b53fb5dbd059ce3","avatarUrl":"/avatars/564c44cf45db794747a96f79f30ecd91.svg","isPro":false,"fullname":"Liu Songhua","user":"Huage001","type":"user"},{"_id":"64d3841af8082bf19b71915b","avatarUrl":"/avatars/c258222a77cd17257cec27407c964f4b.svg","isPro":false,"fullname":"Lincoln","user":"LincolnZed","type":"user"},{"_id":"619f9755da83161f25840698","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/619f9755da83161f25840698/FM421pE1mz5v1YhrxA8ZA.jpeg","isPro":false,"fullname":"Muhammad Umair","user":"umair894","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"686ccb41f10ff747aaea3f13","name":"Shanghai-Jiao-Tong-University-SAI","fullname":"Shanghai Jiao Tong University SAI","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/686cc925516900c6865a1d56/KmvImL_J2ZEHdVFumR3pO.jpeg"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.00079.md","query":{}}">
LeapTalk: Breaking the Latency-Quality Trade-off in Talking Head Generation
Published on Jul 29
· Submitted by zrx on Aug 4 Abstract
Long-form and real-time talking-head generation remains challenging due to a latency-quality trade-off: inefficient multi-step diffusion prohibits streaming generation, whereas real-time autoregressive approaches suffer from error accumulation and identity drift. To address this drawback, we propose LeapTalk, a novel framework that achieves stable and real-time talking-head generation with a single forward step, scaling to arbitrarily long videos. At the heart of our approach lies a single-step bridge distillation scheme. On the one hand, departing from the conventional noise-to-data paradigm, we introduce a data-to-data transport formulation based on a Brownian bridge. Anchored by a persistent reference, this strategy effectively mitigates identity drift and enhances long-term temporal stability. On the other hand, to enable smooth knowledge transfer from a pre-trained diffusion teacher to the student bridge model, we explore a heterogeneous distillation framework with an SNR-aligned time transformation Φ(τ), which bridges the functional discrepancy between the two models. Moreover, we propose an audio-driven classifier-free guidance mechanism to maintain fine-grained lip synchronization under extreme step reduction. Extensive experiments demonstrate that our method achieves high-fidelity and temporally consistent video generation with only 1 step at up to 200 FPS, significantly outperforming existing approaches in both efficiency and stability. Project Page: https://zhangrongxiang.github.io/leaptalk-page/
Community
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2608.00079 in a model README.md to link it from this page.
Cite arxiv.org/abs/2608.00079 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2608.00079 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.