Hugging Face Daily Papers · September 9, 2026 · 4 min read

Omni Interaction Agent Technical Report

Mirrored from Hugging Face Daily Papers for archival readability. Support the source by reading on the original site.

Like Read original ↗

Demo: <a href=\"https://omni-interaction-gander.github.io/Omni-Interaction-Agent/\" rel=\"nofollow\">https://omni-interaction-gander.github.io/Omni-Interaction-Agent/</a></p>\n","updatedAt":"2026-09-09T06:36:47.535Z","author":{"_id":"65afc920091a8ca32b1b0e96","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65afc920091a8ca32b1b0e96/2FxZXNbe7FNQngX5BDzxQ.jpeg","fullname":"jishengpeng","name":"novateur","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":28,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.47899407148361206},"editors":["novateur"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/65afc920091a8ca32b1b0e96/2FxZXNbe7FNQngX5BDzxQ.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2609.08977","authors":[{"_id":"6aa0e3aad0174964227beda7","name":"Orantqing","hidden":false},{"_id":"6aa0e3aad0174964227beda8","user":{"_id":"65afc920091a8ca32b1b0e96","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65afc920091a8ca32b1b0e96/2FxZXNbe7FNQngX5BDzxQ.jpeg","isPro":false,"fullname":"jishengpeng","user":"novateur","type":"user","name":"novateur"},"name":"Shengpeng Ji","status":"claimed_verified","statusLastChangedAt":"2026-09-09T08:45:04.700Z","hidden":false},{"_id":"6aa0e3aad0174964227beda9","name":"Junlong Tong","hidden":false},{"_id":"6aa0e3aad0174964227bedaa","name":"Jialong Zuo","hidden":false},{"_id":"6aa0e3aad0174964227bedab","name":"Dongjie Fu","hidden":false},{"_id":"6aa0e3aad0174964227bedac","name":"Di Cao","hidden":false},{"_id":"6aa0e3aad0174964227bedad","name":"Yangzhuo Li","hidden":false},{"_id":"6aa0e3aad0174964227bedae","name":"Shangda Wu","hidden":false},{"_id":"6aa0e3aad0174964227bedaf","name":"Franz","hidden":false},{"_id":"6aa0e3aad0174964227bedb0","name":"Evan","hidden":false},{"_id":"6aa0e3aad0174964227bedb1","name":"Theron Veyra","hidden":false},{"_id":"6aa0e3aad0174964227bedb2","name":"Changhao Pan","hidden":false},{"_id":"6aa0e3aad0174964227bedb3","name":"Jingyu Lu","hidden":false},{"_id":"6aa0e3aad0174964227bedb4","name":"Dongchao Yang","hidden":false},{"_id":"6aa0e3aad0174964227bedb5","name":"Zhifei Xie","hidden":false},{"_id":"6aa0e3aad0174964227bedb6","name":"Yang Tan","hidden":false},{"_id":"6aa0e3aad0174964227bedb7","name":"Xiaoyu Shen","hidden":false},{"_id":"6aa0e3aad0174964227bedb8","name":"Xiaoda Yang","hidden":false},{"_id":"6aa0e3aad0174964227bedb9","name":"Wenfu Wang","hidden":false},{"_id":"6aa0e3aad0174964227bedba","name":"Teddysun","hidden":false},{"_id":"6aa0e3aad0174964227bedbb","name":"Steveyves","hidden":false},{"_id":"6aa0e3aad0174964227bedbc","name":"Zhou Zhao","hidden":false},{"_id":"6aa0e3aad0174964227bedbd","name":"Bryanytian","hidden":false}],"publishedAt":"2026-09-08T00:00:00.000Z","submittedOnDailyAt":"2026-09-09T00:00:00.000Z","title":"Omni Interaction Agent Technical Report","submittedOnDailyBy":{"_id":"6039478ab3ecf716b1a5fd4d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg","isPro":true,"fullname":"taesiri","user":"taesiri","type":"user","name":"taesiri"},"summary":"In this work, we present Gander, an end-to-end model that unifies omni perception, realtime interaction, and agentic capabilities within a single framework. In contrast to turn-based conventional paradigms, Gander continuously receives streaming inputs across multiple modalities, including video, speech, and text, enabling natural full-duplex interaction in both everyday conversations and complex workflow-oriented agent scenarios. Users can interrupt the model at any time, while the model can also proactively provide intermediate feedback or ask follow up questions. To natively support these capabilities, Gander adopts two key architectural designs: 1) It employs a Cerebellum-Brain collaborative framework, in which the Cerebellum is responsible for realtime interaction and omni conversational capabilities, while the Brain handles complex reasoning and higher-level agentic tasks. The two components interact continuously through tool calling and the agent orchestration runtime. 2) The Cerebellum is built upon a streaming Thinker-Talker architecture, user inputs and model outputs are further flattened into an ordered token stream at the chunk level, providing a unified representation for low latency, continuous interaction. We conduct comprehensive evaluations of Gander across four dimensions: conversational ability, omni understanding, interactive capability, and agentic intelligence. Internal human evaluations demonstrate that Gander maintains the natural and expressive spoken dialogue capabilities of SOTA open source models while achieving competitive performance in omni interaction. Gander also demonstrates robustness in challenging real-world scenarios, including background noise interference, multi-party interactions, and backchannel communication. We release Gander together with its models, code, and data to facilitate further research and development in the community.","upvotes":88,"discussionId":"6aa0e3aad0174964227bedbe","projectPage":"https://omni-interaction-gander.github.io/Omni-Interaction-Agent/","githubRepo":"https://github.com/Omni-Interaction-Gander/Omni-Interaction-Agent","githubRepoAddedBy":"user","ai_summary":"Gander is an end-to-end framework that integrates continuous multi-modal streaming, real-time full-duplex interaction, and agentic reasoning through a Cerebellum-Brain architecture and a chunk-level token stream design.","ai_keywords":["Cerebellum-Brain collaborative framework","streaming Thinker-Talker architecture","token stream","tool calling","agent orchestration runtime","omni perception","full-duplex interaction"],"ai_summary_model":"thinkingmachines/Inkling-Small","githubStars":18,"organization":{"_id":"6645f953c39288df638dbdd5","name":"Tencent-Hunyuan","fullname":"Tencent Hunyuan","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/62d22496c58f969c152bcefd/woKSjt2wXvBNKussyYPsa.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6039478ab3ecf716b1a5fd4d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg","isPro":true,"fullname":"taesiri","user":"taesiri","type":"user"},{"_id":"6a9e616abbf41edd89fdf47a","avatarUrl":"/avatars/014403a18e0f187613f387e2406b1b31.svg","isPro":false,"fullname":"Orant Chen","user":"Gander-Omni","type":"user"},{"_id":"65afc920091a8ca32b1b0e96","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65afc920091a8ca32b1b0e96/2FxZXNbe7FNQngX5BDzxQ.jpeg","isPro":false,"fullname":"jishengpeng","user":"novateur","type":"user"},{"_id":"6645ea5638f0db40582bddcf","avatarUrl":"/avatars/216aeb4d365e28dff484cc275f9f90d7.svg","isPro":false,"fullname":"Yifu Chen","user":"1f","type":"user"},{"_id":"68c11b721e258d133a1d4f2b","avatarUrl":"/avatars/a21daea3c17fec6b817ad6e3fe0dda4d.svg","isPro":false,"fullname":"Jade","user":"wqif","type":"user"},{"_id":"66aa04f0b49f572327eaf5de","avatarUrl":"/avatars/7c28b38c54abcc183bf87a67cf030e4c.svg","isPro":false,"fullname":"sure","user":"theSure","type":"user"},{"_id":"655c2dc713309a611ba2f796","avatarUrl":"/avatars/4de401bba2ab9ac3b67f7a58adba0efb.svg","isPro":false,"fullname":"Jialong Zuo","user":"jlking","type":"user"},{"_id":"65f1b29b6b1729e7298b5bf7","avatarUrl":"/avatars/453ee0576a0b97e6fd68551976bc1084.svg","isPro":false,"fullname":"Zhang Yiang","user":"No1res","type":"user"},{"_id":"68440867f4aaf3d86250daf6","avatarUrl":"/avatars/304a2f18c4d861b557cb734196a6adfd.svg","isPro":false,"fullname":"1f","user":"eve1f","type":"user"},{"_id":"6aa0eb6e2c61145cd76d500e","avatarUrl":"/avatars/9bdc4a91d7673894fcbe9e2dbf09c142.svg","isPro":false,"fullname":"Neural","user":"Spiking","type":"user"},{"_id":"67e4165efc59d8ce68f577c8","avatarUrl":"/avatars/472201827ad292fe6632dd487921928d.svg","isPro":false,"fullname":"xie","user":"filicosTHU","type":"user"},{"_id":"68120a1375e6e2d3c078cc5b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Xh1AQCiYFggk-AjIT-gcB.png","isPro":false,"fullname":"yangrui","user":"yrainbow","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":3,"organization":{"_id":"6645f953c39288df638dbdd5","name":"Tencent-Hunyuan","fullname":"Tencent Hunyuan","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/62d22496c58f969c152bcefd/woKSjt2wXvBNKussyYPsa.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2609/2609.08977.md","query":{}}">

Papers

arxiv:2609.08977

Omni Interaction Agent Technical Report

Published on Sep 8

· Submitted by

taesiri on Sep 9

#3 Paper of the day

Tencent Hunyuan

Upvote

Authors:

Shengpeng Ji ,

Abstract

Gander is an end-to-end framework that integrates continuous multi-modal streaming, real-time full-duplex interaction, and agentic reasoning through a Cerebellum-Brain architecture and a chunk-level token stream design.

Generated by thinkingmachines/Inkling-Small

In this work, we present Gander, an end-to-end model that unifies omni perception, realtime interaction, and agentic capabilities within a single framework. In contrast to turn-based conventional paradigms, Gander continuously receives streaming inputs across multiple modalities, including video, speech, and text, enabling natural full-duplex interaction in both everyday conversations and complex workflow-oriented agent scenarios. Users can interrupt the model at any time, while the model can also proactively provide intermediate feedback or ask follow up questions. To natively support these capabilities, Gander adopts two key architectural designs: 1) It employs a Cerebellum-Brain collaborative framework, in which the Cerebellum is responsible for realtime interaction and omni conversational capabilities, while the Brain handles complex reasoning and higher-level agentic tasks. The two components interact continuously through tool calling and the agent orchestration runtime. 2) The Cerebellum is built upon a streaming Thinker-Talker architecture, user inputs and model outputs are further flattened into an ordered token stream at the chunk level, providing a unified representation for low latency, continuous interaction. We conduct comprehensive evaluations of Gander across four dimensions: conversational ability, omni understanding, interactive capability, and agentic intelligence. Internal human evaluations demonstrate that Gander maintains the natural and expressive spoken dialogue capabilities of SOTA open source models while achieving competitive performance in omni interaction. Gander also demonstrates robustness in challenging real-world scenarios, including background noise interference, multi-party interactions, and backchannel communication. We release Gander together with its models, code, and data to facilitate further research and development in the community.