We release our data, code and prompt templates in <a href=\"https://github.com/yingpengma/NCP-Bench\" rel=\"nofollow\">https://github.com/yingpengma/NCP-Bench</a>.</p>\n","updatedAt":"2026-08-12T06:26:40.771Z","author":{"_id":"67cbb9e7457db2efeff387a0","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/jx4Bjb_mTO3bh9UmMS7Bi.png","fullname":"Yingpeng Ma","name":"mayingpeng","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7709180116653442},"editors":["mayingpeng"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/jx4Bjb_mTO3bh9UmMS7Bi.png"],"reactions":[],"isReport":false}},{"id":"6a7d36691e70b55d42ccebf3","author":{"_id":"6086838b19137b3a6ba760e7","avatarUrl":"/avatars/d63eea3e39b22c6e65b82c28192696f1.svg","fullname":"Jianhao Yan","name":"Elliott","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":9,"isUserFollowing":false},"createdAt":"2026-08-13T03:13:45.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Sharing our new work at ICML 2026 on interactive narrative and long-horizon consistency ☀️ We explore the question: when players can act freely in an open-ended way, as interactions get longer and events accumulate, can the story world still hold together?\n\nFor example, the story has already established that “the only key is on the guard.” The player might choose to steal it, break down the door by force, or lie that they already have the key. The model needs to respond reasonably to the player’s actions, while ensuring that established facts do not conflict with the next turn and that later must-happen plot points do not vanish into thin air. We call this ability Narrative Commitment Preservation: preserving established facts and completing the key plot points that must be advanced later.\n\nTo this end, we built NCP-Bench from 100 classic movie synopses. Each story is designed as an interactive environment containing an initial set of facts, key events during the story, and plot goals to achieve. On NCP-Bench, Narrator Agent and Player Agent interact for up to 100 turns, and we check at each turn for contradictions, whether the model genuinely responds to the player’s actions, and whether key plot points are completed.\n\nOur evaluation shows that long-range interaction remains difficult for current models: the best-performing model, GPT-5.2, remains conflict-free in only 42% of cases after 20 turns; across models, 40% to 68% of interactions ultimately end in factual conflicts. Under the 100-turn limit, the number of cases that make it through the entire interaction without problems while completing all key plot goals is in the single digits.","html":"<p>Sharing our new work at ICML 2026 on interactive narrative and long-horizon consistency ☀️ We explore the question: when players can act freely in an open-ended way, as interactions get longer and events accumulate, can the story world still hold together?</p>\n<p>For example, the story has already established that “the only key is on the guard.” The player might choose to steal it, break down the door by force, or lie that they already have the key. The model needs to respond reasonably to the player’s actions, while ensuring that established facts do not conflict with the next turn and that later must-happen plot points do not vanish into thin air. We call this ability Narrative Commitment Preservation: preserving established facts and completing the key plot points that must be advanced later.</p>\n<p>To this end, we built NCP-Bench from 100 classic movie synopses. Each story is designed as an interactive environment containing an initial set of facts, key events during the story, and plot goals to achieve. On NCP-Bench, Narrator Agent and Player Agent interact for up to 100 turns, and we check at each turn for contradictions, whether the model genuinely responds to the player’s actions, and whether key plot points are completed.</p>\n<p>Our evaluation shows that long-range interaction remains difficult for current models: the best-performing model, GPT-5.2, remains conflict-free in only 42% of cases after 20 turns; across models, 40% to 68% of interactions ultimately end in factual conflicts. Under the 100-turn limit, the number of cases that make it through the entire interaction without problems while completing all key plot goals is in the single digits.</p>\n","updatedAt":"2026-08-13T03:13:45.818Z","author":{"_id":"6086838b19137b3a6ba760e7","avatarUrl":"/avatars/d63eea3e39b22c6e65b82c28192696f1.svg","fullname":"Jianhao Yan","name":"Elliott","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":9,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.935950517654419},"editors":["Elliott"],"editorAvatarUrls":["/avatars/d63eea3e39b22c6e65b82c28192696f1.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.08160","authors":[{"_id":"6a7c10d11653ef87c6af1d90","name":"Yingpeng Ma","hidden":false},{"_id":"6a7c10d11653ef87c6af1d91","name":"Jianhao Yan","hidden":false},{"_id":"6a7c10d11653ef87c6af1d92","name":"Bei Shi","hidden":false},{"_id":"6a7c10d11653ef87c6af1d93","name":"Ka Hou Kam","hidden":false},{"_id":"6a7c10d11653ef87c6af1d94","name":"Runnan Wang","hidden":false},{"_id":"6a7c10d11653ef87c6af1d95","name":"Xuebo Liu","hidden":false},{"_id":"6a7c10d11653ef87c6af1d96","name":"Yulong Chen","hidden":false},{"_id":"6a7c10d11653ef87c6af1d97","name":"Yue Zhang","hidden":false},{"_id":"6a7c10d11653ef87c6af1d98","name":"Derek F. Wong","hidden":false}],"publishedAt":"2026-08-08T00:00:00.000Z","submittedOnDailyAt":"2026-08-13T00:00:00.000Z","title":"Can LLM Agents Stick to the Script? A Benchmark for Long-Horizon Consistency in Interactive Narratives","submittedOnDailyBy":{"_id":"6086838b19137b3a6ba760e7","avatarUrl":"/avatars/d63eea3e39b22c6e65b82c28192696f1.svg","isPro":false,"fullname":"Jianhao Yan","user":"Elliott","type":"user","name":"Elliott"},"summary":"The rapid advancement of Large Language Models (LLMs) is revolutionizing AI for Games by enabling open-ended and fluid interactive storytelling. However, existing research has largely overlooked the critical challenge of maintaining long-horizon logical consistency and narrative integrity against unconstrained user interventions. To address this, we formulate this challenge as Narrative Commitment Preservation (NCP), and take interactive narrative as our testbed. We introduce NCP-Bench, a benchmark of 100 narrative environments derived from movie synopses. Each environment includes a structured narrative specification (trajectory, commitments, and initial facts) that we can automatically check throughout the interaction between the player agent and the narrator agent. Experiments across state-of-the-art LLMs reveal a substantial long-horizon consistency gap: high linguistic quality does not guarantee commitment preservation; even strong models frequently generate logically conflicting content under adversarial interventions, with the best-performing model (GPT-5.2) achieving only 42% survival rate after 20 turns and fact conflict rates ranging from 40% to 68% across models, and only isolated runs satisfying all achievement commitments within the 100-turn limit.","upvotes":25,"discussionId":"6a7c10d11653ef87c6af1d99","githubRepo":"https://github.com/yingpengma/NCP-Bench","githubRepoAddedBy":"user","ai_summary":"The study introduces a benchmark and formalizes narrative commitment preservation to evaluate long-horizon logical consistency in interactive storytelling with large language models.","ai_keywords":["Narrative Commitment Preservation","NCP-Bench","interactive narrative","trajectory","commitments","initial facts","adversarial interventions","survival rate","fact conflict rates","achievement commitments"],"ai_summary_model":"thinkingmachines/Inkling-Small","githubStars":9},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"67cbb9e7457db2efeff387a0","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/jx4Bjb_mTO3bh9UmMS7Bi.png","isPro":false,"fullname":"Yingpeng Ma","user":"mayingpeng","type":"user"},{"_id":"6086838b19137b3a6ba760e7","avatarUrl":"/avatars/d63eea3e39b22c6e65b82c28192696f1.svg","isPro":false,"fullname":"Jianhao Yan","user":"Elliott","type":"user"},{"_id":"6376e7c4cb2e84c60b0c602b","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6376e7c4cb2e84c60b0c602b/3NLrsB8KQPiOEgpF0zGjP.jpeg","isPro":false,"fullname":"YANG SHU","user":"babytreecc","type":"user"},{"_id":"64082c00a7bc7c3865b19a68","avatarUrl":"/avatars/035e8c5405d0f7407d7fca258837d375.svg","isPro":false,"fullname":"Bei SHI","user":"axcin","type":"user"},{"_id":"64c8fb6be761f470612a342a","avatarUrl":"/avatars/d6e1d753f39d9491bc00e2f0169d5e57.svg","isPro":false,"fullname":"xinyi","user":"x1iris","type":"user"},{"_id":"616648c84c0937d31946f21b","avatarUrl":"/avatars/7ca27de5c5116c91ff1db61ba6277ed5.svg","isPro":false,"fullname":"Ziyang","user":"hzy","type":"user"},{"_id":"62495cb96ee7ee6b646db130","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/62495cb96ee7ee6b646db130/UwBXmvcMq7LMvBWUw0xo3.jpeg","isPro":false,"fullname":"Runzhe Zhan","user":"rzzhan","type":"user"},{"_id":"654ce87af0b05673196a9f45","avatarUrl":"/avatars/7b9c854eb98e487e3057479b1c7860ac.svg","isPro":false,"fullname":"Shudong Liu","user":"Sudanl","type":"user"},{"_id":"65697feb9fb2d79a79e14e0a","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65697feb9fb2d79a79e14e0a/wVGaBjn8pQIJneZWSFIwS.jpeg","isPro":false,"fullname":"haodi lei","user":"bingyang-lei","type":"user"},{"_id":"66aca01e33f6b27979856f6f","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/66aca01e33f6b27979856f6f/IyOxv89TudwscGH7tdue3.jpeg","isPro":false,"fullname":"Mingyang Song","user":"hitsmy","type":"user"},{"_id":"63a2a51ef30c464227924fc6","avatarUrl":"/avatars/e109e85abd25b97bb29dbbe007119e34.svg","isPro":false,"fullname":"Haoyu Sun","user":"Mikivis","type":"user"},{"_id":"689ec537196ab997b13dc977","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/689ec537196ab997b13dc977/yXA_pd8ndjBIIg1Hx59QJ.png","isPro":false,"fullname":"Haoran Zhang","user":"zzzhr97","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.08160.md","query":{}}">
Can LLM Agents Stick to the Script? A Benchmark for Long-Horizon Consistency in Interactive Narratives
Abstract
The study introduces a benchmark and formalizes narrative commitment preservation to evaluate long-horizon logical consistency in interactive storytelling with large language models.
The rapid advancement of Large Language Models (LLMs) is revolutionizing AI for Games by enabling open-ended and fluid interactive storytelling. However, existing research has largely overlooked the critical challenge of maintaining long-horizon logical consistency and narrative integrity against unconstrained user interventions. To address this, we formulate this challenge as Narrative Commitment Preservation (NCP), and take interactive narrative as our testbed. We introduce NCP-Bench, a benchmark of 100 narrative environments derived from movie synopses. Each environment includes a structured narrative specification (trajectory, commitments, and initial facts) that we can automatically check throughout the interaction between the player agent and the narrator agent. Experiments across state-of-the-art LLMs reveal a substantial long-horizon consistency gap: high linguistic quality does not guarantee commitment preservation; even strong models frequently generate logically conflicting content under adversarial interventions, with the best-performing model (GPT-5.2) achieving only 42% survival rate after 20 turns and fact conflict rates ranging from 40% to 68% across models, and only isolated runs satisfying all achievement commitments within the 100-turn limit.
Community
Sharing our new work at ICML 2026 on interactive narrative and long-horizon consistency ☀️ We explore the question: when players can act freely in an open-ended way, as interactions get longer and events accumulate, can the story world still hold together?
For example, the story has already established that “the only key is on the guard.” The player might choose to steal it, break down the door by force, or lie that they already have the key. The model needs to respond reasonably to the player’s actions, while ensuring that established facts do not conflict with the next turn and that later must-happen plot points do not vanish into thin air. We call this ability Narrative Commitment Preservation: preserving established facts and completing the key plot points that must be advanced later.
To this end, we built NCP-Bench from 100 classic movie synopses. Each story is designed as an interactive environment containing an initial set of facts, key events during the story, and plot goals to achieve. On NCP-Bench, Narrator Agent and Player Agent interact for up to 100 turns, and we check at each turn for contradictions, whether the model genuinely responds to the player’s actions, and whether key plot points are completed.
Our evaluation shows that long-range interaction remains difficult for current models: the best-performing model, GPT-5.2, remains conflict-free in only 42% of cases after 20 turns; across models, 40% to 68% of interactions ultimately end in factual conflicts. Under the 100-turn limit, the number of cases that make it through the entire interaction without problems while completing all key plot goals is in the single digits.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2608.08160 in a model README.md to link it from this page.
Cite arxiv.org/abs/2608.08160 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2608.08160 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.