Self-Listening for Full-Duplex Speech Models</p>\n","updatedAt":"2026-09-09T13:15:36.882Z","author":{"_id":"637c6703ca8542a0ba900ccb","avatarUrl":"/avatars/288ed63a1efa566c3f01e850c6ba5dd5.svg","fullname":"Wang","name":"Benyou","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":11,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.6774232983589172},"editors":["Benyou"],"editorAvatarUrls":["/avatars/288ed63a1efa566c3f01e850c6ba5dd5.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2609.05592","authors":[{"_id":"6aa154a5d8c54e38c0a369bd","user":{"_id":"68074965bb30e830e3ed633e","avatarUrl":"/avatars/1dc03c7d88fd63defac3d21261275d4e.svg","isPro":false,"fullname":"Xuanning Zhou","user":"shinning721","type":"user","name":"shinning721"},"name":"Xuanning Zhou","status":"claimed_verified","statusLastChangedAt":"2026-09-09T16:45:04.630Z","hidden":false},{"_id":"6aa154a5d8c54e38c0a369be","name":"Junyi Ao","hidden":false},{"_id":"6aa154a5d8c54e38c0a369bf","name":"Xiaotong Liu","hidden":false},{"_id":"6aa154a5d8c54e38c0a369c0","name":"Tom Ko","hidden":false},{"_id":"6aa154a5d8c54e38c0a369c1","name":"Benyou Wang","hidden":false},{"_id":"6aa154a5d8c54e38c0a369c2","name":"Haizhou Li","hidden":false}],"publishedAt":"2026-09-04T00:00:00.000Z","submittedOnDailyAt":"2026-09-09T00:00:00.000Z","title":"What Did I Just Say? Self-Listening for Full-Duplex Speech Models","submittedOnDailyBy":{"_id":"637c6703ca8542a0ba900ccb","avatarUrl":"/avatars/288ed63a1efa566c3f01e850c6ba5dd5.svg","isPro":false,"fullname":"Wang","user":"Benyou","type":"user","name":"Benyou"},"summary":"Full-duplex spoken language models can listen and speak simultaneously, enabling them to handle interruptions and backchannels in human conversation. However, text generation, speech synthesis, and audio playback proceed asynchronously. As a result, what a model believes it has said may not match what has actually been played to the user. We refer to the problem of recovering from an interruption while remaining aware of the model's realized speech as anchor interruption. To address this problem, we propose Self-Listening, a full-duplex modeling approach that interleaves user speech, model text, and the model's played speech. By feeding the realized speech output back to the model as an input stream, self-listening grounds interruption recovery in what the user has actually heard. We further introduce AnchorSpeech, a collection with homogeneous training and test splits for tracking which items of structured ordered responses have actually been spoken. AnchorSpeech-test evaluates whether a model can respond consistently with the last completed item before an interruption. Experiments show that, compared with full-duplex baselines, models equipped with self-listening mechanism achieve better anchoring performance.","upvotes":16,"discussionId":"6aa154a5d8c54e38c0a369c3","githubRepo":"https://github.com/FreedomIntelligence/LoopSpeech","githubRepoAddedBy":"user","ai_summary":"Self-Listening grounds interruption recovery in full-duplex spoken language models by feeding realized speech back as input, improving consistency with actually spoken responses.","ai_keywords":["full-duplex spoken language models","self-listening","anchor interruption","realized speech","AnchorSpeech"],"ai_summary_model":"thinkingmachines/Inkling-Small","githubStars":1,"organization":{"_id":"640e87ec3830fd441c2dd85a","name":"FreedomIntelligence","fullname":"FreedomAI","avatar":"https://www.gravatar.com/avatar/ad73607eb900e074a2e973297f74962b?d=retro&size=100"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"68074965bb30e830e3ed633e","avatarUrl":"/avatars/1dc03c7d88fd63defac3d21261275d4e.svg","isPro":false,"fullname":"Xuanning Zhou","user":"shinning721","type":"user"},{"_id":"637c6703ca8542a0ba900ccb","avatarUrl":"/avatars/288ed63a1efa566c3f01e850c6ba5dd5.svg","isPro":false,"fullname":"Wang","user":"Benyou","type":"user"},{"_id":"69fff76e75f0c7ac77e6f101","avatarUrl":"/avatars/5d4a18cec51075a9a3540b6ddff4f52d.svg","isPro":false,"fullname":"HUANG","user":"Ulrica91","type":"user"},{"_id":"67548a30481dae41e8adc7ed","avatarUrl":"/avatars/fa6577759934ca3b2171ed12cfcc8a24.svg","isPro":false,"fullname":"Zijie ZHANG","user":"zijie-zhang","type":"user"},{"_id":"64f1a34f2c5c8b767916447e","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64f1a34f2c5c8b767916447e/uak2CsMAnxW8q4dwyAOBN.jpeg","isPro":false,"fullname":"Zhenyang Cai","user":"Eric3200","type":"user"},{"_id":"67061847e343e345b777d574","avatarUrl":"/avatars/6b6f35aa563dcf32937c4b6a3cdd870e.svg","isPro":false,"fullname":"Zhiyu","user":"zylin1","type":"user"},{"_id":"68c6a75b68632ba6d823ba25","avatarUrl":"/avatars/56f3097995baa24cf473b907eea139bf.svg","isPro":false,"fullname":"aluyilin2020","user":"aluyilin2020","type":"user"},{"_id":"63ca949b04c979828315389d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63ca949b04c979828315389d/HS5xWNAYjjHeyAAwWJ11l.jpeg","isPro":false,"fullname":"wangrongsheng","user":"wangrongsheng","type":"user"},{"_id":"65fbdbc8fc9132a2dfd67c8f","avatarUrl":"/avatars/9e404fca5f7b53d49e1aa73d525f834d.svg","isPro":false,"fullname":"Minghao Wu","user":"magicc0nch","type":"user"},{"_id":"61ad83716da53246bd6ac428","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/61ad83716da53246bd6ac428/B8b87v78C6c2s9WCVkcmt.jpeg","isPro":false,"fullname":"Junyi Ao","user":"ajyy","type":"user"},{"_id":"656ae4088fb1ddf0d5ec9ac5","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/656ae4088fb1ddf0d5ec9ac5/cRz98086YIPhS8lJRcOf2.jpeg","isPro":false,"fullname":"Junxian Li","user":"Duke-de-Artois","type":"user"},{"_id":"68a2cb600a7ccf97b6f12d0a","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/68a2cb600a7ccf97b6f12d0a/oHubNvZ7Sil_W0PDldX-u.jpeg","isPro":false,"fullname":"Yihang Du","user":"YihDu","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"640e87ec3830fd441c2dd85a","name":"FreedomIntelligence","fullname":"FreedomAI","avatar":"https://www.gravatar.com/avatar/ad73607eb900e074a2e973297f74962b?d=retro&size=100"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2609/2609.05592.md","query":{}}">
What Did I Just Say? Self-Listening for Full-Duplex Speech Models
Published on Sep 4
· Submitted by Wang on Sep 9 Abstract
Self-Listening grounds interruption recovery in full-duplex spoken language models by feeding realized speech back as input, improving consistency with actually spoken responses.
Full-duplex spoken language models can listen and speak simultaneously, enabling them to handle interruptions and backchannels in human conversation. However, text generation, speech synthesis, and audio playback proceed asynchronously. As a result, what a model believes it has said may not match what has actually been played to the user. We refer to the problem of recovering from an interruption while remaining aware of the model's realized speech as anchor interruption. To address this problem, we propose Self-Listening, a full-duplex modeling approach that interleaves user speech, model text, and the model's played speech. By feeding the realized speech output back to the model as an input stream, self-listening grounds interruption recovery in what the user has actually heard. We further introduce AnchorSpeech, a collection with homogeneous training and test splits for tracking which items of structured ordered responses have actually been spoken. AnchorSpeech-test evaluates whether a model can respond consistently with the last completed item before an interruption. Experiments show that, compared with full-duplex baselines, models equipped with self-listening mechanism achieve better anchoring performance.
Community
Self-Listening for Full-Duplex Speech Models
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2609.05592 in a model README.md to link it from this page.
Cite arxiv.org/abs/2609.05592 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2609.05592 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.