Dataset: <a href=\"https://huggingface.co/datasets/ShawnLi02/JigShape-Train\">https://huggingface.co/datasets/ShawnLi02/JigShape-Train</a></p>\n","updatedAt":"2026-08-12T03:46:57.683Z","author":{"_id":"62c5947524171688a9feb992","avatarUrl":"/avatars/5a151713b9eae8dc566f5957acee3475.svg","fullname":"Franck Dernoncourt","name":"Franck-Dernoncourt","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":14,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.6742885708808899},"editors":["Franck-Dernoncourt"],"editorAvatarUrls":["/avatars/5a151713b9eae8dc566f5957acee3475.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2607.27670","authors":[{"_id":"6a7beb221653ef87c6af1c85","name":"Shawn Li","hidden":false},{"_id":"6a7beb221653ef87c6af1c86","name":"Wei Yang","hidden":false},{"_id":"6a7beb221653ef87c6af1c87","name":"Jike Zhong","hidden":false},{"_id":"6a7beb221653ef87c6af1c88","name":"Jiate Li","hidden":false},{"_id":"6a7beb221653ef87c6af1c89","name":"Jiawei Yang","hidden":false},{"_id":"6a7beb221653ef87c6af1c8a","name":"You Qin","hidden":false},{"_id":"6a7beb221653ef87c6af1c8b","name":"Ryan Rossi","hidden":false},{"_id":"6a7beb221653ef87c6af1c8c","user":{"_id":"62c5947524171688a9feb992","avatarUrl":"/avatars/5a151713b9eae8dc566f5957acee3475.svg","isPro":false,"fullname":"Franck Dernoncourt","user":"Franck-Dernoncourt","type":"user","name":"Franck-Dernoncourt"},"name":"Franck Dernoncourt","status":"claimed_verified","statusLastChangedAt":"2026-08-12T08:45:05.134Z","hidden":false},{"_id":"6a7beb221653ef87c6af1c8d","name":"Roger Zimmermann","hidden":false},{"_id":"6a7beb221653ef87c6af1c8e","name":"Yue Wang","hidden":false},{"_id":"6a7beb221653ef87c6af1c8f","name":"Zhengzhong Tu","hidden":false},{"_id":"6a7beb221653ef87c6af1c90","name":"Vicente Ordonez","hidden":false},{"_id":"6a7beb221653ef87c6af1c91","name":"Mohit Bansal","hidden":false},{"_id":"6a7beb221653ef87c6af1c92","name":"Yue Zhao","hidden":false}],"publishedAt":"2026-08-04T00:00:00.000Z","submittedOnDailyAt":"2026-08-12T00:00:00.000Z","title":"JigShape: Evaluating Visual-Geometric Reasoning in VLMs through Jigsaw Puzzles","submittedOnDailyBy":{"_id":"62c5947524171688a9feb992","avatarUrl":"/avatars/5a151713b9eae8dc566f5957acee3475.svg","isPro":false,"fullname":"Franck Dernoncourt","user":"Franck-Dernoncourt","type":"user","name":"Franck-Dernoncourt"},"summary":"Jigsaw puzzle solving requires jointly reasoning about visual content and geometric constraints, yet existing benchmarks use rectangular cuts that create ambiguous ground truth in texture-repeated regions. We introduce \\ours{}, a benchmark with tab-and-blank interlocking pieces where geometric constraints provide strong local compatibility requirements that, combined with visual content, yield unambiguous ground truth. Across 95K instances at four grid densities (4times4 to 16times16), we find that zero-shot VLMs largely lack geometric reasoning: only one of five frontier models (GPT-5.5) exceeds random baseline on 4times4 puzzles, while all others perform at chance level. While supervised fine-tuning achieves >97\\% on 4times4, all models collapse on larger grids: GPT-5.5 drops from 70\\% to near-random on 8times8, and even fine-tuned models fall below 5\\% on 12times12. This ``scaling cliff'' suggests current architectures cannot maintain consistent constraint satisfaction as the number of pieces increases. establishes scalable geometric reasoning as an open challenge for vision-language models.","upvotes":3,"discussionId":"6a7beb231653ef87c6af1c93","projectPage":"https://huggingface.co/datasets/ShawnLi02/JigShape-Train","ai_summary":"A new jigsaw benchmark with interlocking pieces reveals that vision-language models fail at geometric reasoning and suffer a sharp performance drop as puzzle size increases.","ai_keywords":["vision-language models","geometric reasoning","jigsaw puzzles","tab-and-blank interlocking pieces","supervised fine-tuning","scaling cliff"],"ai_summary_model":"thinkingmachines/Inkling-Small"},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"62c5947524171688a9feb992","avatarUrl":"/avatars/5a151713b9eae8dc566f5957acee3475.svg","isPro":false,"fullname":"Franck Dernoncourt","user":"Franck-Dernoncourt","type":"user"},{"_id":"699e9c5b9b93bc6afbfabbf7","avatarUrl":"/avatars/aca99ea58c13c506000c67953ebfb8a0.svg","isPro":false,"fullname":"Taylor Levi","user":"donghao56","type":"user"},{"_id":"63ac5701c21e60a3e9b58aa7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63ac5701c21e60a3e9b58aa7/g6EX7diOpuA94R2ab-rZC.png","isPro":true,"fullname":"Dipankar Sarkar","user":"dipankarsarkar","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2607/2607.27670.md","query":{}}">

Get this paper in your agent:

Don't have the latest CLI?

Cite arxiv.org/abs/2607.27670 in a model README.md to link it from this page.

Cite arxiv.org/abs/2607.27670 in a dataset README.md to link it from this page.

Cite arxiv.org/abs/2607.27670 in a Space README.md to link it from this page.

Add this paper to a collection to link it from this page.