EMNLP Findings 2026</p>\n","updatedAt":"2026-09-04T09:03:56.830Z","author":{"_id":"668aeec93d34648deb2aad43","avatarUrl":"/avatars/684fc4b7fdcc9424940ca84865cc3fa4.svg","fullname":"Nan Jiang","name":"jiangnanhugo","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.6607754826545715},"editors":["jiangnanhugo"],"editorAvatarUrls":["/avatars/684fc4b7fdcc9424940ca84865cc3fa4.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.30391","authors":[{"_id":"6a9a86808f7c3b755723971f","name":"Zhuoran Lu","hidden":false},{"_id":"6a9a86808f7c3b7557239720","name":"Yangyang Yu","hidden":false},{"_id":"6a9a86808f7c3b7557239721","name":"Zhuoyan Li","hidden":false},{"_id":"6a9a86808f7c3b7557239722","name":"Yibo Meng","hidden":false},{"_id":"6a9a86808f7c3b7557239723","name":"Nan Jiang","hidden":false},{"_id":"6a9a86808f7c3b7557239724","name":"Chengxi Zang","hidden":false},{"_id":"6a9a86808f7c3b7557239725","name":"Jie Gao","hidden":false},{"_id":"6a9a86808f7c3b7557239726","name":"Ziang Xiao","hidden":false}],"publishedAt":"2026-08-31T00:00:00.000Z","submittedOnDailyAt":"2026-09-04T00:00:00.000Z","title":"Using Grounded Theory for Agent Behavior Analysis at Scale","submittedOnDailyBy":{"_id":"668aeec93d34648deb2aad43","avatarUrl":"/avatars/684fc4b7fdcc9424940ca84865cc3fa4.svg","isPro":false,"fullname":"Nan Jiang","user":"jiangnanhugo","type":"user","name":"jiangnanhugo"},"summary":"Understanding agent behavior requires methods that scale to thousands of trajectories and surface new patterns in long, often unfamiliar tasks where pre-built classifiers fall short. We propose to bring grounded theory into agent trajectory analysis: a six-decade-old qualitative method from the social sciences, with a principled saturation criterion and an auditable trail from data to theory. We propose AutoTraceGT (Automated Trace analysis through Grounded Theory), the first multi-agent pipeline that automates grounded theory on agent trajectories. It iteratively performs open, axial, and theoretical coding until saturation, producing a behavioral taxonomy tailored to each task. Across six trajectory corpora, AutoTraceGT produces codebooks that recover 73-91 percent of the failure modes in human-annotated taxonomies and surface additional patterns that those taxonomies miss. The emergent theoretical narrative aligns with prior expert accounts. Used as a deductive feature space, the codebook outperforms zero-shot and few-shot LLM baselines on downstream failure prediction. These results suggest Grounded Theory offers a scalable analytic tool for ML researchers and agent developers studying what agents actually do.","upvotes":0,"discussionId":"6a9a86818f7c3b7557239727","projectPage":"https://nan-jiang-group.github.io/AutoTraceGT/","ai_summary":"AutoTraceGT automates grounded theory coding on agent trajectories to build task-specific behavioral taxonomies that recover and extend human failure-mode annotations for downstream prediction.","ai_keywords":["grounded theory","open coding","axial coding","theoretical coding","saturation","AutoTraceGT","behavioral taxonomy","failure prediction","deductive feature space"],"ai_summary_model":"thinkingmachines/Inkling-Small","organization":{"_id":"6400300fe7767a89533fa2c0","name":"Purdue","fullname":"Purdue University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1677733893266-64002f32cafc9d54986439cb.jpeg"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[],"acceptLanguages":["en"],"organization":{"_id":"6400300fe7767a89533fa2c0","name":"Purdue","fullname":"Purdue University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1677733893266-64002f32cafc9d54986439cb.jpeg"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.30391.md","query":{}}">
Using Grounded Theory for Agent Behavior Analysis at Scale
Abstract
AutoTraceGT automates grounded theory coding on agent trajectories to build task-specific behavioral taxonomies that recover and extend human failure-mode annotations for downstream prediction.
Understanding agent behavior requires methods that scale to thousands of trajectories and surface new patterns in long, often unfamiliar tasks where pre-built classifiers fall short. We propose to bring grounded theory into agent trajectory analysis: a six-decade-old qualitative method from the social sciences, with a principled saturation criterion and an auditable trail from data to theory. We propose AutoTraceGT (Automated Trace analysis through Grounded Theory), the first multi-agent pipeline that automates grounded theory on agent trajectories. It iteratively performs open, axial, and theoretical coding until saturation, producing a behavioral taxonomy tailored to each task. Across six trajectory corpora, AutoTraceGT produces codebooks that recover 73-91 percent of the failure modes in human-annotated taxonomies and surface additional patterns that those taxonomies miss. The emergent theoretical narrative aligns with prior expert accounts. Used as a deductive feature space, the codebook outperforms zero-shot and few-shot LLM baselines on downstream failure prediction. These results suggest Grounded Theory offers a scalable analytic tool for ML researchers and agent developers studying what agents actually do.
Community
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2608.30391 in a model README.md to link it from this page.
Cite arxiv.org/abs/2608.30391 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2608.30391 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.