The paper presents Audio-Interaction, a unified streaming model for real-time audio interaction, accompanied by the SoundFlow framework, the StreamAudio-2M dataset, and new benchmarks for proactive audio intervention.</p>\n<p> Video: <a href=\"https://www.youtube.com/watch?v=4YuBkMm1cmU\" rel=\"nofollow\">https://www.youtube.com/watch?v=4YuBkMm1cmU</a></p>\n","updatedAt":"2026-06-04T02:21:34.754Z","author":{"_id":"6039478ab3ecf716b1a5fd4d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg","fullname":"taesiri","name":"taesiri","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":310,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7668321132659912},"editors":["taesiri"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.05121","authors":[{"_id":"6a20e0ff15100c5272a8468d","name":"Zhifei Xie","hidden":false},{"_id":"6a20e0ff15100c5272a8468e","name":"Zihang Liu","hidden":false},{"_id":"6a20e0ff15100c5272a8468f","name":"Ze An","hidden":false},{"_id":"6a20e0ff15100c5272a84690","name":"Xiaobin Hu","hidden":false},{"_id":"6a20e0ff15100c5272a84691","name":"Yue Liao","hidden":false},{"_id":"6a20e0ff15100c5272a84692","name":"Ziyang Ma","hidden":false},{"_id":"6a20e0ff15100c5272a84693","name":"Dongchao Yang","hidden":false},{"_id":"6a20e0ff15100c5272a84694","name":"Mingbao Lin","hidden":false},{"_id":"6a20e0ff15100c5272a84695","name":"Deheng Ye","hidden":false},{"_id":"6a20e0ff15100c5272a84696","name":"Shuicheng Yan","hidden":false},{"_id":"6a20e0ff15100c5272a84697","name":"Chunyan Miao","hidden":false}],"publishedAt":"2026-06-03T00:00:00.000Z","submittedOnDailyAt":"2026-06-04T00:00:00.000Z","title":"Audio Interaction Model","submittedOnDailyBy":{"_id":"6039478ab3ecf716b1a5fd4d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg","isPro":true,"fullname":"taesiri","user":"taesiri","type":"user","name":"taesiri"},"summary":"Audio is an inherently interactive modality, yet today's Large Audio Language Models (LALMs) are offline, and streaming audio models each handle only a single task such as streaming ASR or voice chatting. It is time to unify them into one online LALM: a model that, through an always-on perceive-decide-respond loop, listens to sound, environment, and instructions in real time and reacts on the fly. We formalize this regime as the Audio Interaction Model, and realize it with Audio-Interaction, a unified streaming model that retains offline task execution while adding online general audio instruction following, from dialogue to full voice chatting, deciding when to respond from the semantics of the stream. To enable this, we propose SoundFlow, a framework that instantiates the perceive-decide-respond loop end to end, from data to training to deployment, through streaming-native data construction, comprehension-aware training, and asynchronous low-latency inference for stable real-time interaction. We further construct StreamAudio-2M, a 2.6M-item streaming corpus spanning 7 fundamental abilities and 28 sub-tasks, and Proactive-Sound-Bench for evaluating proactive audio intervention. Across 8 benchmarks, Audio-Interaction preserves competitive performance on mainstream audio tasks while unlocking capabilities inaccessible to offline LALMs, including real-time ASR, streaming audio instruction following, and proactive help.","upvotes":5,"discussionId":"6a20e10015100c5272a84698","projectPage":"https://xzf-thu.github.io/Audio-Interaction/","ai_summary":"A unified streaming audio model is developed that combines offline task execution with real-time audio instruction following through an end-to-end framework supporting multiple audio interaction capabilities.","ai_keywords":["Large Audio Language Models","streaming audio models","perceive-decide-respond loop","Audio Interaction Model","SoundFlow","streaming-native data construction","comprehension-aware training","asynchronous low-latency inference","StreamAudio-2M","Proactive-Sound-Bench"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct"},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6039478ab3ecf716b1a5fd4d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg","isPro":true,"fullname":"taesiri","user":"taesiri","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"63c5d43ae2804cb2407e4d43","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1673909278097-noauth.png","isPro":false,"fullname":"xziayro","user":"xziayro","type":"user"},{"_id":"664d930f4b870dd167473c1c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/664d930f4b870dd167473c1c/TXVEPGvkhftdI_xE1mluu.jpeg","isPro":false,"fullname":"Andy Guan","user":"andytonglove","type":"user"},{"_id":"69bd10643a4a2f915a1d0902","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/Hatkqlq2G2dcVvr7_kHL7.jpeg","isPro":false,"fullname":"渡辺結翔","user":"JSANCHEZ18","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.05121.md"}">

Get this paper in your agent:

Don't have the latest CLI?

Cite arxiv.org/abs/2606.05121 in a model README.md to link it from this page.

Cite arxiv.org/abs/2606.05121 in a dataset README.md to link it from this page.

Cite arxiv.org/abs/2606.05121 in a Space README.md to link it from this page.

Add this paper to a collection to link it from this page.