PRISM is a benchmarking framework that evaluates LLM peer reviewers against human experts across four scientifically grounded dimensions—depth of analysis, novelty assessment, flaw identification, and constructiveness. Applied to five leading automated reviewer systems on 1,000 papers from ICLR, ICML, and NeurIPS, PRISM finds that LLMs can match or exceed humans on individual dimensions, but no single system sustains this across all four simultaneously—each excels in a distinct niche while exhibiting structured blind spots invisible to aggregate metrics.</p>\n","updatedAt":"2026-05-29T09:54:29.836Z","author":{"_id":"64f7647204852a0233e5b888","avatarUrl":"/avatars/d40faf8dfc9fe4f82df1ab6a5ab6ae0e.svg","fullname":"Duy A Nguyen","name":"anhduy0911","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8921948671340942},"editors":["anhduy0911"],"editorAvatarUrls":["/avatars/d40faf8dfc9fe4f82df1ab6a5ab6ae0e.svg"],"reactions":[],"isReport":false}},{"id":"6a1a4133039cfb1a5d9b1568","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":359,"isUserFollowing":false},"createdAt":"2026-05-30T01:45:23.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [PRAIB: Peer Review AI Benchmark of Behaviour of LLM-Assisted Reviewing](https://huggingface.co/papers/2605.29815) (2026)\n* [CoCoReviewBench: A Completeness- and Correctness-Oriented Benchmark for AI Reviewers](https://huggingface.co/papers/2605.07905) (2026)\n* [NovBench: Evaluating Large Language Models on Academic Paper Novelty Assessment](https://huggingface.co/papers/2604.11543) (2026)\n* [Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the Future](https://huggingface.co/papers/2604.27924) (2026)\n* [GoodPoint: Learning Constructive Scientific Paper Feedback from Author Responses](https://huggingface.co/papers/2604.11924) (2026)\n* [On the limits and opportunities of AI reviewers: Reviewing the reviews of Nature-family papers with 45 expert scientists](https://huggingface.co/papers/2605.20668) (2026)\n* [Beyond Rating: A Comprehensive Evaluation and Benchmark for AI Reviews](https://huggingface.co/papers/2604.19502) (2026)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"<p>This is an automated message from the <a href=\"https://huggingface.co/librarian-bots\">Librarian Bot</a>. I found the following papers similar to this paper. </p>\n<p>The following papers were recommended by the Semantic Scholar API </p>\n<ul>\n<li><a href=\"https://huggingface.co/papers/2605.29815\">PRAIB: Peer Review AI Benchmark of Behaviour of LLM-Assisted Reviewing</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.07905\">CoCoReviewBench: A Completeness- and Correctness-Oriented Benchmark for AI Reviewers</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2604.11543\">NovBench: Evaluating Large Language Models on Academic Paper Novelty Assessment</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2604.27924\">Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the Future</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2604.11924\">GoodPoint: Learning Constructive Scientific Paper Feedback from Author Responses</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.20668\">On the limits and opportunities of AI reviewers: Reviewing the reviews of Nature-family papers with 45 expert scientists</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2604.19502\">Beyond Rating: A Comprehensive Evaluation and Benchmark for AI Reviews</a> (2026)</li>\n</ul>\n<p> Please give a thumbs up to this comment if you found it helpful!</p>\n<p> If you want recommendations for any Paper on Hugging Face checkout <a href=\"https://huggingface.co/spaces/librarian-bots/recommend_similar_papers\">this</a> Space</p>\n<p> You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: <code><span class=\"SVELTE_PARTIAL_HYDRATER contents\" data-target=\"UserMention\" data-props=\"{&quot;user&quot;:&quot;librarian-bot&quot;}\"><span class=\"inline-block\"><span class=\"contents\"><a href=\"/librarian-bot\">@<span class=\"underline\">librarian-bot</span></a></span> </span></span> recommend</code></p>\n","updatedAt":"2026-05-30T01:45:23.430Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":359,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7702999711036682},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2605.26730","authors":[{"_id":"6a19615337c01891a4ac9c58","name":"Ngoc Phan Phuoc Loc","hidden":false},{"_id":"6a19615337c01891a4ac9c59","name":"Toan Huynh La Viet","hidden":false},{"_id":"6a19615337c01891a4ac9c5a","name":"Thanh Tran Khanh","hidden":false},{"_id":"6a19615337c01891a4ac9c5b","name":"Duy A Nguyen","hidden":false},{"_id":"6a19615337c01891a4ac9c5c","name":"Tuan Anh Nguyen Pham","hidden":false},{"_id":"6a19615337c01891a4ac9c5d","name":"Thanh Nguyen","hidden":false},{"_id":"6a19615337c01891a4ac9c5e","name":"Nitesh V. Chawla","hidden":false},{"_id":"6a19615337c01891a4ac9c5f","name":"Wray Buntine","hidden":false},{"_id":"6a19615337c01891a4ac9c60","name":"Kok-Seng Wong","hidden":false},{"_id":"6a19615337c01891a4ac9c61","name":"Khoa D. Doan","hidden":false},{"_id":"6a19615337c01891a4ac9c62","name":"Binh T. Nguyen","hidden":false}],"publishedAt":"2026-05-27T00:00:00.000Z","submittedOnDailyAt":"2026-05-29T00:00:00.000Z","title":"PRISM: A Multi-Dimensional Benchmark for Evaluating LLM Peer Reviewers","submittedOnDailyBy":{"_id":"64f7647204852a0233e5b888","avatarUrl":"/avatars/d40faf8dfc9fe4f82df1ab6a5ab6ae0e.svg","isPro":false,"fullname":"Duy A Nguyen","user":"anhduy0911","type":"user","name":"anhduy0911"},"summary":"The rapid growth in submissions to machine learning venues has strained the scientific peer-review system and intensified interest in LLM-based automated peer reviewers. However, how good these systems are actually, especially compared to human reviewers at catching scientific gaps, remains poorly understood. In this work, we introduce PRISM (Peer Review Intelligence via Structured Multi-dimensional assessment), a benchmarking framework that evaluates review quality across four dimensions: Depth of Analysis, Novelty Assessment,Flaw Identification & Major Issues Prioritization, and Multi-dimensional Constructiveness. Unlike most existing evaluations based on surface-level metrics like ROUGE and BLEU, or unconstrained LLM-as-a-judge prompting that conflates fluency with rigor, PRISM grounds each dimension in argument mining, retrieval-augmented verification, and consensus-based scoring. We apply PRISM to benchmark five leading automated reviewer systems and human reviewers on a stratified corpus of reviews from ICLR, ICML, and NeurIPS. The results reveal that LLMs can match or beat human reviewers on individual dimensions: comparable depth of analysis, stronger novelty verification, and highly accurate critique prioritization. However, no single system consistently matches the balanced performance of the human baseline across all dimensions at once. Each exhibits a distinct specialization profile with characteristic blind spots -- failure modes that aggregate metrics miss entirely. The implication is that LLM reviewers are best understood as targeted supplements to human review, effective within specific dimensions, but unreliable as standalone replacements. Our demo and key results can be found at https://khanhthanhdev.github.io/prism-page/.","upvotes":8,"discussionId":"6a19615337c01891a4ac9c63","projectPage":"https://prism-benchmark.github.io/","ai_summary":"PRISM evaluates automated peer review systems across multiple dimensions using argument mining and retrieval-augmented verification, revealing that while LLMs match human performance in specific areas, no system consistently equals human reviewers across all evaluation criteria.","ai_keywords":["peer review intelligence","structured multi-dimensional assessment","argument mining","retrieval-augmented verification","consensus-based scoring","automated reviewer systems","human reviewers","ICLR","ICML","NeurIPS"],"organization":{"_id":"64c8c778b8685df8003c3a94","name":"VinUniversity","fullname":"VinUniversity","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/64c8c69a1c23fb9a2be7c2c3/Mjk72FkD7VNasXNnOQDCb.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"69f95278e184017219d57694","avatarUrl":"/avatars/f6eff78328add6dc24492b2d7dbc8db8.svg","isPro":false,"fullname":"Anonymous researcher","user":"anoyresearcher","type":"user"},{"_id":"64f7647204852a0233e5b888","avatarUrl":"/avatars/d40faf8dfc9fe4f82df1ab6a5ab6ae0e.svg","isPro":false,"fullname":"Duy A Nguyen","user":"anhduy0911","type":"user"},{"_id":"687639140037d77c0a977b97","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/bCYvEwzNKNjeoMa2ppfJ9.jpeg","isPro":false,"fullname":"Trung Hieu Do","user":"blmppes","type":"user"},{"_id":"6707359cc818ba2f6578d1ec","avatarUrl":"/avatars/f5e96487e0996a9bb2e7c2180bef855a.svg","isPro":false,"fullname":"Tran Anh Chuong","user":"Jimmytrn154","type":"user"},{"_id":"66b0b313b4dd007544fa9491","avatarUrl":"/avatars/fe772b7045d8daac677bc5a5afae9c9e.svg","isPro":false,"fullname":"Ngoc Phan","user":"Twind","type":"user"},{"_id":"6690fbec5f4e78f111600507","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6690fbec5f4e78f111600507/jJHOi9AzNaWAP0w8LGUxj.jpeg","isPro":false,"fullname":"Tran Khanh Thanh","user":"thanhkt","type":"user"},{"_id":"66589869c6ed5d7b6c35f1d5","avatarUrl":"/avatars/2f858f5ad0fa46c70e7d4581c6ed4663.svg","isPro":false,"fullname":"Khoa Doan","user":"khoadoan106","type":"user"},{"_id":"648961d150c003881f1a10c3","avatarUrl":"/avatars/1eb3784c39f7ced2e952d11a410933ae.svg","isPro":false,"fullname":"Harshita Sharma","user":"hdsharma","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"64c8c778b8685df8003c3a94","name":"VinUniversity","fullname":"VinUniversity","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/64c8c69a1c23fb9a2be7c2c3/Mjk72FkD7VNasXNnOQDCb.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2605/2605.26730.md"}">

Get this paper in your agent:

Don't have the latest CLI?

Cite arxiv.org/abs/2605.26730 in a model README.md to link it from this page.

Cite arxiv.org/abs/2605.26730 in a dataset README.md to link it from this page.

Cite arxiv.org/abs/2605.26730 in a Space README.md to link it from this page.

Add this paper to a collection to link it from this page.