Conversational AI has reached billions of users, yet existing datasets capture only what people say, not what they think. ThoughtTrace is the first large-scale dataset that pairs real-world multi-turn human–AI conversations with users' self-reported thoughts: their reasons for sending prompts and reactions to assistant responses. ThoughtTrace comprises 1,058 users, 2,155 conversations, and 10,174 thought annotations collected across 20 language models.</p>\n","updatedAt":"2026-05-21T00:02:04.233Z","author":{"_id":"64beb69801f1983a86a05de2","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64beb69801f1983a86a05de2/tFyCoqZ6gT8NWkZfuncID.jpeg","fullname":"Chuanyang Jin","name":"Chuanyang-Jin","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":10,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9305336475372314},"editors":["Chuanyang-Jin"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/64beb69801f1983a86a05de2/tFyCoqZ6gT8NWkZfuncID.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2605.20087","authors":[{"_id":"6a0e4b23164dbbc68a26c3c4","name":"Chuanyang Jin","hidden":false},{"_id":"6a0e4b23164dbbc68a26c3c5","name":"Binze Li","hidden":false},{"_id":"6a0e4b23164dbbc68a26c3c6","name":"Haopeng Xie","hidden":false},{"_id":"6a0e4b23164dbbc68a26c3c7","name":"Cathy Mengying Fang","hidden":false},{"_id":"6a0e4b23164dbbc68a26c3c8","name":"Tianjian Li","hidden":false},{"_id":"6a0e4b23164dbbc68a26c3c9","name":"Shayne Longpre","hidden":false},{"_id":"6a0e4b23164dbbc68a26c3ca","name":"Hongxiang Gu","hidden":false},{"_id":"6a0e4b23164dbbc68a26c3cb","name":"Maximillian Chen","hidden":false},{"_id":"6a0e4b23164dbbc68a26c3cc","name":"Tianmin Shu","hidden":false}],"publishedAt":"2026-05-19T00:00:00.000Z","submittedOnDailyAt":"2026-05-20T00:00:00.000Z","title":"ThoughtTrace: Understanding User Thoughts in Real-World LLM Interactions","submittedOnDailyBy":{"_id":"64beb69801f1983a86a05de2","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64beb69801f1983a86a05de2/tFyCoqZ6gT8NWkZfuncID.jpeg","isPro":false,"fullname":"Chuanyang Jin","user":"Chuanyang-Jin","type":"user","name":"Chuanyang-Jin"},"summary":"Conversational AI has now reached billions of users, yet existing datasets capture only what people say, not what they think. We introduce ThoughtTrace, the first large-scale dataset that pairs real-world multi-turn human--AI conversations with users' self-reported thoughts: their reasons for sending prompts and reactions to assistant responses. ThoughtTrace comprises 1,058 users, 2,155 conversations, 17,058 turns, and 10,174 thought annotations collected across 20 language models. Our analysis shows that ThoughtTrace captures long-horizon, topically diverse interactions, and that thoughts are semantically distinct from messages, difficult for frontier LLMs to infer from context, diverse in content, and tied to conversation stages. We further demonstrate the utility of thoughts for downstream modeling. First, thoughts improve user-behavior prediction as inference-time context. Second, thought-guided rewrites provide fine-grained alignment signals for training personalized assistants. Together, ThoughtTrace establishes user thoughts as a new data modality for studying the cognitive dynamics behind human--AI interaction and provides a foundation for building assistants that better understand and adapt to users' latent goals, preferences, and needs.","upvotes":3,"discussionId":"6a0e4b24164dbbc68a26c3cd","projectPage":"https://thoughttrace-project.github.io/","githubRepo":"https://github.com/thoughttrace-project/thoughttrace","githubRepoAddedBy":"user","ai_summary":"ThoughtTrace presents a large-scale dataset pairing human-AI conversations with self-reported thoughts, enabling improved user behavior prediction and personalized assistant training through thought-guided rewrites.","ai_keywords":["conversational AI","multi-turn conversations","large-scale dataset","user-behavior prediction","thought-guided rewrites","personalized assistants","cognitive dynamics","latent goals","preferences","needs"],"githubStars":3,"organization":{"_id":"653945b47ba797097a7b4eab","name":"JohnsHopkins","fullname":"Johns Hopkins University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/653944e58e687a41625a4694/qqHzBOarppVrUuZbbjqwh.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"64beb69801f1983a86a05de2","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64beb69801f1983a86a05de2/tFyCoqZ6gT8NWkZfuncID.jpeg","isPro":false,"fullname":"Chuanyang Jin","user":"Chuanyang-Jin","type":"user"},{"_id":"66cbea552400073af31bdf7f","avatarUrl":"/avatars/5de3c4da60c6097b6bd4ce245113e1c2.svg","isPro":false,"fullname":"Social Cognitive AI Lab at JHU","user":"SCAI-JHU","type":"user"},{"_id":"643a97d83b71e2b0cc1bb59a","avatarUrl":"/avatars/ef03a43e8e67475b5a8c1c143cf96dde.svg","isPro":true,"fullname":"Yifan Yin","user":"yifanyin11","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"653945b47ba797097a7b4eab","name":"JohnsHopkins","fullname":"Johns Hopkins University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/653944e58e687a41625a4694/qqHzBOarppVrUuZbbjqwh.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2605/2605.20087.md"}">

Get this paper in your agent:

Don't have the latest CLI?

Cite arxiv.org/abs/2605.20087 in a model README.md to link it from this page.

Cite arxiv.org/abs/2605.20087 in a Space README.md to link it from this page.

Add this paper to a collection to link it from this page.