Large language models are increasingly deployed as agents that reason over documents rather than answer from parametric knowledge. We study archive-grounded reasoning: locating sparse evidence across a large, messy collection of workplace files, reconciling inconsistent terminology, units, and time conventions, and computing an answer. Existing benchmarks address only parts of this setting and none jointly stresses archive-groundedness, agentic exploration, and cross-domain coverage. We introduce Agora, a benchmark pairing 362 questions with eight domain collections of 9,664 authentic documents and 372M tokens, far exceeding any model's context window, so agents must explore deliberately rather than scan exhaustively. Agora is built by an agentic pipeline combining cross-document task synthesis, leakage-preventing obfuscation, and difficulty filtering. Evaluating eight models, we find the task far from solved: even the strongest reaches only 59.4% accuracy, with notable variation across domains.<br><a href=\"https://cdn-uploads.huggingface.co/production/uploads/638ef0b0c67af472d31674a6/PDgrpYgLo3bkxREzeaTBV.png\" rel=\"nofollow\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/638ef0b0c67af472d31674a6/PDgrpYgLo3bkxREzeaTBV.png\" alt=\"image\"></a></p>\n<p><a href=\"https://cdn-uploads.huggingface.co/production/uploads/638ef0b0c67af472d31674a6/otCc6A2okDH54m9fbr_sb.png\" rel=\"nofollow\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/638ef0b0c67af472d31674a6/otCc6A2okDH54m9fbr_sb.png\" alt=\"image\"></a></p>\n<p><a href=\"https://cdn-uploads.huggingface.co/production/uploads/638ef0b0c67af472d31674a6/YQKDpqT4a7z_u2w3LyV-I.png\" rel=\"nofollow\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/638ef0b0c67af472d31674a6/YQKDpqT4a7z_u2w3LyV-I.png\" alt=\"image\"></a></p>\n","updatedAt":"2026-06-24T12:04:31.571Z","author":{"_id":"638ef0b0c67af472d31674a6","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/638ef0b0c67af472d31674a6/zXQjC3DdY3jpVkATkpms6.png","fullname":"Honglin Guo","name":"KYLN24","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":6,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8466829657554626},"editors":["KYLN24"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/638ef0b0c67af472d31674a6/zXQjC3DdY3jpVkATkpms6.png"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.24526","authors":[{"_id":"6a3bc6e15ac9fb074498498d","name":"Honglin Guo","hidden":false},{"_id":"6a3bc6e15ac9fb074498498e","name":"Qi Zhang","hidden":false},{"_id":"6a3bc6e15ac9fb074498498f","name":"Yu Zhang","hidden":false},{"_id":"6a3bc6e15ac9fb0744984990","name":"Weijie Li","hidden":false},{"_id":"6a3bc6e15ac9fb0744984991","name":"Rui Zheng","hidden":false},{"_id":"6a3bc6e15ac9fb0744984992","name":"Zhikai Lei","hidden":false},{"_id":"6a3bc6e15ac9fb0744984993","name":"Qiyuan Peng","hidden":false},{"_id":"6a3bc6e15ac9fb0744984994","name":"Zhiheng Xi","hidden":false},{"_id":"6a3bc6e15ac9fb0744984995","name":"Tao Gui","hidden":false},{"_id":"6a3bc6e15ac9fb0744984996","name":"Qi Zhang","hidden":false}],"publishedAt":"2026-06-23T00:00:00.000Z","submittedOnDailyAt":"2026-06-24T00:00:00.000Z","title":"AGORA: An Archive-Grounded Benchmark for Agentic Workplace Document Reasoning","submittedOnDailyBy":{"_id":"638ef0b0c67af472d31674a6","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/638ef0b0c67af472d31674a6/zXQjC3DdY3jpVkATkpms6.png","isPro":false,"fullname":"Honglin Guo","user":"KYLN24","type":"user","name":"KYLN24"},"summary":"Large language models are increasingly deployed as agents that reason over documents rather than answer from parametric knowledge. We study archive-grounded reasoning: locating sparse evidence across a large, messy collection of workplace files, reconciling inconsistent terminology, units, and time conventions, and computing an answer. Existing benchmarks address only parts of this setting and none jointly stresses archive-groundedness, agentic exploration, and cross-domain coverage. We introduce Agora, a benchmark pairing 362 questions with eight domain collections of 9,664 authentic documents and 372M tokens, far exceeding any model's context window, so agents must explore deliberately rather than scan exhaustively. Agora is built by an agentic pipeline combining cross-document task synthesis, leakage-preventing obfuscation, and difficulty filtering. Evaluating eight models, we find the task far from solved: even the strongest reaches only 59.4% accuracy, with notable variation across domains.","upvotes":0,"discussionId":"6a3bc6e15ac9fb0744984997","ai_summary":"Large language models face challenges in archive-grounded reasoning tasks involving evidence retrieval and synthesis across diverse document collections, with performance varying significantly across domains.","ai_keywords":["large language models","archive-grounded reasoning","evidence retrieval","cross-document task synthesis","leakage-preventing obfuscation","difficulty filtering","agentic pipeline","domain collections","document synthesis"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct"},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[],"acceptLanguages":["en"],"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.24526.md","query":{}}">
AGORA: An Archive-Grounded Benchmark for Agentic Workplace Document Reasoning
Authors: ,
,
,
,
,
,
,
,
,
Abstract
Large language models face challenges in archive-grounded reasoning tasks involving evidence retrieval and synthesis across diverse document collections, with performance varying significantly across domains.
Large language models are increasingly deployed as agents that reason over documents rather than answer from parametric knowledge. We study archive-grounded reasoning: locating sparse evidence across a large, messy collection of workplace files, reconciling inconsistent terminology, units, and time conventions, and computing an answer. Existing benchmarks address only parts of this setting and none jointly stresses archive-groundedness, agentic exploration, and cross-domain coverage. We introduce Agora, a benchmark pairing 362 questions with eight domain collections of 9,664 authentic documents and 372M tokens, far exceeding any model's context window, so agents must explore deliberately rather than scan exhaustively. Agora is built by an agentic pipeline combining cross-document task synthesis, leakage-preventing obfuscation, and difficulty filtering. Evaluating eight models, we find the task far from solved: even the strongest reaches only 59.4% accuracy, with notable variation across domains.
Community
Large language models are increasingly deployed as agents that reason over documents rather than answer from parametric knowledge. We study archive-grounded reasoning: locating sparse evidence across a large, messy collection of workplace files, reconciling inconsistent terminology, units, and time conventions, and computing an answer. Existing benchmarks address only parts of this setting and none jointly stresses archive-groundedness, agentic exploration, and cross-domain coverage. We introduce Agora, a benchmark pairing 362 questions with eight domain collections of 9,664 authentic documents and 372M tokens, far exceeding any model's context window, so agents must explore deliberately rather than scan exhaustively. Agora is built by an agentic pipeline combining cross-document task synthesis, leakage-preventing obfuscation, and difficulty filtering. Evaluating eight models, we find the task far from solved: even the strongest reaches only 59.4% accuracy, with notable variation across domains.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2606.24526 in a model README.md to link it from this page.
Cite arxiv.org/abs/2606.24526 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2606.24526 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.