Hugging Face Daily Papers · September 15, 2026 · 4 min read

Omni-Streaming Thinking

Mirrored from Hugging Face Daily Papers for archival readability. Support the source by reading on the original site.

Like Read original ↗

<a href=\"https://enjundu.com/blog/ost/\" rel=\"nofollow\">https://enjundu.com/blog/ost/</a></p>\n","updatedAt":"2026-09-15T01:41:07.243Z","author":{"_id":"66decf61f9971122eec44dc8","avatarUrl":"/avatars/ffd1bf114f2fffc1f9a2ffe5964543f3.svg","fullname":"Enjun Du","name":"EnjunDu","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":3,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.5934708714485168},"editors":["EnjunDu"],"editorAvatarUrls":["/avatars/ffd1bf114f2fffc1f9a2ffe5964543f3.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2609.15128","authors":[{"_id":"6aa8a2105dd4cb9b4cc02497","name":"Enjun Du","hidden":false},{"_id":"6aa8a2105dd4cb9b4cc02498","name":"Siyi Liu","hidden":false},{"_id":"6aa8a2105dd4cb9b4cc02499","name":"Ziyu Zheng","hidden":false},{"_id":"6aa8a2105dd4cb9b4cc0249a","name":"Jingyu Li","hidden":false},{"_id":"6aa8a2105dd4cb9b4cc0249b","name":"Yiwen Guo","hidden":false},{"_id":"6aa8a2105dd4cb9b4cc0249c","name":"Yongqi Zhang","hidden":false},{"_id":"6aa8a2105dd4cb9b4cc0249d","name":"Difan Zou","hidden":false}],"publishedAt":"2026-09-14T00:00:00.000Z","submittedOnDailyAt":"2026-09-15T00:00:00.000Z","title":"Omni-Streaming Thinking","submittedOnDailyBy":{"_id":"66decf61f9971122eec44dc8","avatarUrl":"/avatars/ffd1bf114f2fffc1f9a2ffe5964543f3.svg","isPro":false,"fullname":"Enjun Du","user":"EnjunDu","type":"user","name":"EnjunDu"},"summary":"Streaming omni-modal models must decide what and when to answer from the video chunks and synchronized audio observed so far. Visual cues often support an interpretation before an utterance or sound event is complete. If that interpretation enters memory as a fact, later reasoning can keep relaying it even after audio contradicts it. We call this failure premature cross-modal commitment. We propose Omni-Streaming Thinking (OST), which generates structured outputs that include evidence observed so far, forecasts of future evidence, and claims based on this evidence. Each claim is initially marked as pending and linked to a future verification interval. Audio and visual evidence are stored separately, and OST checks a claim against the evidence from the specified modality at the end of the verification interval. When contradictory evidence is detected, a refutation process reduces the influence of the claim and its dependent states, and then guides a state update using the new evidence. An answer gate decides whether the answer-critical claims meet the conditions for giving a response. Using a frozen Qwen3-Omni-30B-A3B-Instruct backbone with lightweight adaptation, OST outperforms the strongest open baselines on five streaming and audio-visual benchmarks by more than 10% relative on average. We also introduce OST-DiagBench, which holds video fixed and edits audio to test agreement, absence, contradiction, coexistence, and subtitle-speech conflict. OST reaches d-prime = 2.95, compared with at most 1.38 for open baselines, while reducing vision-induced auditory hallucinations.","upvotes":12,"discussionId":"6aa8a2105dd4cb9b4cc0249e","projectPage":"https://enjundu.com/blog/ost/","ai_summary":"Omni-Streaming Thinking improves streaming omni-modal reasoning by deferring claims until cross-modal verification, reducing premature commitment and auditory hallucinations.","ai_keywords":["omni-modal models","streaming","cross-modal commitment","Omni-Streaming Thinking","verification interval","refutation process","answer gate","Qwen3-Omni-30B-A3B-Instruct","OST-DiagBench","d-prime","vision-induced auditory hallucinations"],"ai_summary_model":"thinkingmachines/Inkling-Small"},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"66decf61f9971122eec44dc8","avatarUrl":"/avatars/ffd1bf114f2fffc1f9a2ffe5964543f3.svg","isPro":false,"fullname":"Enjun Du","user":"EnjunDu","type":"user"},{"_id":"6a2da6c8ca070ee12c6e396c","avatarUrl":"/avatars/0355287dcabaa67dbc7f0b10b87451f9.svg","isPro":false,"fullname":"Joe Mama","user":"JoeMama123123123","type":"user"},{"_id":"687f853bb39262ba84f3eeff","avatarUrl":"/avatars/cdfc44fde8237f08f10192553fe5a075.svg","isPro":false,"fullname":"Junhao Shen","user":"shenjunhao","type":"user"},{"_id":"65745569839aa08899ea5d27","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/4X8waDwiphbfKZySrYlFy.jpeg","isPro":false,"fullname":"Kailin Jiang","user":"kailinjiang","type":"user"},{"_id":"665abce1d599adace6d66674","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/665abce1d599adace6d66674/pd6IiwPkbT4ocOySleXga.jpeg","isPro":false,"fullname":"Selen Su","user":"selensu","type":"user"},{"_id":"67fab241a0b4ccba867ef93c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Dm7LsQd9ZPzFh71i_o0Ke.png","isPro":false,"fullname":"yanxt","user":"Yanxt-re","type":"user"},{"_id":"675f94f62654de2575256b88","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/XB6tJLgcmYI8Ha6hreyYY.png","isPro":false,"fullname":"YubinGuo","user":"shockstove","type":"user"},{"_id":"65df4f5af3efe60b06ea1409","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65df4f5af3efe60b06ea1409/u-9mnMU8bNXvyQ8z1CG5m.png","isPro":false,"fullname":"Xinye Li","user":"asdfo123","type":"user"},{"_id":"66a246cae22bfd8d72560696","avatarUrl":"/avatars/97cec177a0eb0a8465e3532b888ef1d4.svg","isPro":false,"fullname":"zrchen","user":"zrchen03","type":"user"},{"_id":"668fcd97495674543600db0b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/JfBeJ6gSXe379G9ejjrLR.jpeg","isPro":false,"fullname":"Gao Mingqi","user":"EchoMinkki","type":"user"},{"_id":"656c3503e0ff1cebe95e5730","avatarUrl":"/avatars/e9e6ecde1ca79833bfe23df51ee0dc68.svg","isPro":false,"fullname":"Tsuki","user":"Tsukihjy","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2609/2609.15128.md","query":{}}">

Papers

arxiv:2609.15128

Omni-Streaming Thinking

Published on Sep 14

· Submitted by

Enjun Du on Sep 15

Upvote

Authors:

Abstract

Omni-Streaming Thinking improves streaming omni-modal reasoning by deferring claims until cross-modal verification, reducing premature commitment and auditory hallucinations.

Generated by thinkingmachines/Inkling-Small

Streaming omni-modal models must decide what and when to answer from the video chunks and synchronized audio observed so far. Visual cues often support an interpretation before an utterance or sound event is complete. If that interpretation enters memory as a fact, later reasoning can keep relaying it even after audio contradicts it. We call this failure premature cross-modal commitment. We propose Omni-Streaming Thinking (OST), which generates structured outputs that include evidence observed so far, forecasts of future evidence, and claims based on this evidence. Each claim is initially marked as pending and linked to a future verification interval. Audio and visual evidence are stored separately, and OST checks a claim against the evidence from the specified modality at the end of the verification interval. When contradictory evidence is detected, a refutation process reduces the influence of the claim and its dependent states, and then guides a state update using the new evidence. An answer gate decides whether the answer-critical claims meet the conditions for giving a response. Using a frozen Qwen3-Omni-30B-A3B-Instruct backbone with lightweight adaptation, OST outperforms the strongest open baselines on five streaming and audio-visual benchmarks by more than 10% relative on average. We also introduce OST-DiagBench, which holds video fixed and edits audio to test agreement, absence, contradiction, coexistence, and subtitle-speech conflict. OST reaches d-prime = 2.95, compared with at most 1.38 for open baselines, while reducing vision-induced auditory hallucinations.