SOCO is a benchmark for evaluating structured, part-level understanding in vision and multimodal foundation models through semantic correspondence. It provides a taxonomy of correspondence types, functionally meaningful keypoint annotations across 100 categories, over 1M correspondence pairs, and language descriptions for evaluating LVLMs. Experiments show that current vision backbones encode semantic structure but struggle with cross-category correspondence and object-part position, while LVLMs perform better at text-prompted localization than visual-reference matching. SOCO also shows that semantic correspondence is a stronger predictor of dense downstream performance than ImageNet classification.</p>\n","updatedAt":"2026-06-02T12:36:54.004Z","author":{"_id":"65ddd4fe54a95b60de6af7d6","avatarUrl":"/avatars/f0f4af777662d3fad62b70d0e185a116.svg","fullname":"Olaf Dünkel","name":"odunkel","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8061365485191345},"editors":["odunkel"],"editorAvatarUrls":["/avatars/f0f4af777662d3fad62b70d0e185a116.svg"],"reactions":[],"isReport":false}},{"id":"6a1f8a9de548c640d0eee9ca","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":360,"isUserFollowing":false},"createdAt":"2026-06-03T01:59:57.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [MARCO: Navigating the Unseen Space of Semantic Correspondence](https://huggingface.co/papers/2604.18267) (2026)\n* [Geometry Matters: 3D Foundation Priors for Learning Semantic Correspondence](https://huggingface.co/papers/2605.30093) (2026)\n* [Beyond Accuracy: Benchmarking Cross-Task Consistency in Unified Multimodal Models](https://huggingface.co/papers/2604.25072) (2026)\n* [SENSE: Stereo OpEN Vocabulary SEmantic Segmentation](https://huggingface.co/papers/2604.15946) (2026)\n* [SEMAGIC: Learning Semantically Consistent Deformable 3D Representations from In-the-Wild Images](https://huggingface.co/papers/2605.27938) (2026)\n* [Learning to Perceive\"Where\": Spatial Pretext Tasks for Robust Self-Supervised Learning](https://huggingface.co/papers/2605.09963) (2026)\n* [Beyond 3D VQAs: Injecting 3D Spatial Priors into Vision-Language Models for Enhanced Geometric Reasoning](https://huggingface.co/papers/2605.30231) (2026)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"<p>This is an automated message from the <a href=\"https://huggingface.co/librarian-bots\">Librarian Bot</a>. I found the following papers similar to this paper. </p>\n<p>The following papers were recommended by the Semantic Scholar API </p>\n<ul>\n<li><a href=\"https://huggingface.co/papers/2604.18267\">MARCO: Navigating the Unseen Space of Semantic Correspondence</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.30093\">Geometry Matters: 3D Foundation Priors for Learning Semantic Correspondence</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2604.25072\">Beyond Accuracy: Benchmarking Cross-Task Consistency in Unified Multimodal Models</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2604.15946\">SENSE: Stereo OpEN Vocabulary SEmantic Segmentation</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.27938\">SEMAGIC: Learning Semantically Consistent Deformable 3D Representations from In-the-Wild Images</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.09963\">Learning to Perceive\"Where\": Spatial Pretext Tasks for Robust Self-Supervised Learning</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.30231\">Beyond 3D VQAs: Injecting 3D Spatial Priors into Vision-Language Models for Enhanced Geometric Reasoning</a> (2026)</li>\n</ul>\n<p> Please give a thumbs up to this comment if you found it helpful!</p>\n<p> If you want recommendations for any Paper on Hugging Face checkout <a href=\"https://huggingface.co/spaces/librarian-bots/recommend_similar_papers\">this</a> Space</p>\n<p> You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: <code><span class=\"SVELTE_PARTIAL_HYDRATER contents\" data-target=\"UserMention\" data-props=\"{"user":"librarian-bot"}\"><span class=\"inline-block\"><span class=\"contents\"><a href=\"/librarian-bot\">@<span class=\"underline\">librarian-bot</span></a></span> </span></span> recommend</code></p>\n","updatedAt":"2026-06-03T01:59:57.538Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":360,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7250523567199707},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2605.31597","authors":[{"_id":"6a1db3a6808ddbc3c7d43997","user":{"_id":"65ddd4fe54a95b60de6af7d6","avatarUrl":"/avatars/f0f4af777662d3fad62b70d0e185a116.svg","isPro":false,"fullname":"Olaf Dünkel","user":"odunkel","type":"user","name":"odunkel"},"name":"Olaf Dünkel","status":"claimed_verified","statusLastChangedAt":"2026-06-02T12:09:54.820Z","hidden":false},{"_id":"6a1db3a6808ddbc3c7d43998","name":"Basavaraj Sunagad","hidden":false},{"_id":"6a1db3a6808ddbc3c7d43999","name":"Haoran Wang","hidden":false},{"_id":"6a1db3a6808ddbc3c7d4399a","name":"David T. Hoffmann","hidden":false},{"_id":"6a1db3a6808ddbc3c7d4399b","name":"Christian Theobalt","hidden":false},{"_id":"6a1db3a6808ddbc3c7d4399c","name":"Adam Kortylewski","hidden":false}],"publishedAt":"2026-05-29T00:00:00.000Z","submittedOnDailyAt":"2026-06-02T00:00:00.000Z","title":"SOCO: Benchmarking Semantic Object Correspondence in Vision Foundation Models","submittedOnDailyBy":{"_id":"65ddd4fe54a95b60de6af7d6","avatarUrl":"/avatars/f0f4af777662d3fad62b70d0e185a116.svg","isPro":false,"fullname":"Olaf Dünkel","user":"odunkel","type":"user","name":"odunkel"},"summary":"Measuring structured object understanding in vision foundation models remains challenging due to inconsistent evaluation protocols and limited part-level supervision. Semantic correspondence (SC) evaluates this capability by testing whether object parts can be matched across instances and categories under large variations in appearance, viewpoint, and geometry. To enable a systematic SC evaluation, we introduce SOCO, a new benchmark for Semantic Object Correspondence that introduces a taxonomy of correspondence types and provides consistent, functionally meaningful keypoint annotations across 100 categories and over 1M correspondence pairs. In addition, SOCO includes keypoint language descriptions, enabling the evaluation of large vision-language models (LVLMs) and their fine-grained part-level understanding. Comprehensive experiments reveal that (i) vision foundation backbones encode strong semantic structure but transfer correspondences poorly across related categories and only partially capture object-part position, (ii) LVLMs are stronger at text-prompted part localization than at visual-reference cross-image matching, exposing a gap between language-grounded localization and fine-grained visual correspondence, and (iii) correspondence performance predicts performance on dense downstream tasks, including segmentation, tracking, 3D pose estimation, and 3D detection, more strongly than ImageNet classification. Together, these findings position SOCO as a benchmark for structured, part-level representation quality in vision and multimodal foundation models.","upvotes":5,"discussionId":"6a1db3a6808ddbc3c7d4399d","projectPage":"https://genintel.github.io/SOCO/","githubRepo":"https://github.com/GenIntel/OmniProbe","githubRepoAddedBy":"user","ai_summary":"Semantic Object Correspondence (SOCO) benchmark evaluates structured object understanding in vision models through consistent part-level annotations and keypoint descriptions, revealing gaps between language-grounded localization and visual correspondence while demonstrating strong prediction of downstream task performance.","ai_keywords":["semantic correspondence","vision foundation models","vision-language models","part-level supervision","keypoints","correspondence types","structured object understanding","downstream tasks","segmentation","tracking","3D pose estimation","3D detection","ImageNet classification"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","githubStars":4,"organization":{"_id":"6a1560761c24e63f40df826c","name":"GenIntelLab","fullname":"Generative Intelligence Lab","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/6a0ad49529c6a75e98eddb0f/QAh9LhDbOGW5_k-WIPx04.jpeg"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"65ddd4fe54a95b60de6af7d6","avatarUrl":"/avatars/f0f4af777662d3fad62b70d0e185a116.svg","isPro":false,"fullname":"Olaf Dünkel","user":"odunkel","type":"user"},{"_id":"632227fc48b40e273b5e9bdd","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/632227fc48b40e273b5e9bdd/Lw1VIRy_moOqYqZOOvSiH.jpeg","isPro":false,"fullname":"Artur Jesslen","user":"Arturjssln","type":"user"},{"_id":"66147ae0e2ba01441b59b437","avatarUrl":"/avatars/b0fc7d3f00c5a51eacf2ffb5fa8e1508.svg","isPro":false,"fullname":"Henry Wong","user":"krumo","type":"user"},{"_id":"692432f01f66351978f150b3","avatarUrl":"/avatars/9e09c73f54123c732368b059e08e3daf.svg","isPro":false,"fullname":"Leon Begiristain Ribo","user":"Lehoi","type":"user"},{"_id":"69bcb352c72e2028614f65b4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/3iiIxHpJSn0YtPXpykYeR.png","isPro":false,"fullname":"Luo Yuxuan","user":"asherlnc2026","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"6a1560761c24e63f40df826c","name":"GenIntelLab","fullname":"Generative Intelligence Lab","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/6a0ad49529c6a75e98eddb0f/QAh9LhDbOGW5_k-WIPx04.jpeg"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2605/2605.31597.md"}">
SOCO: Benchmarking Semantic Object Correspondence in Vision Foundation Models
Abstract
Semantic Object Correspondence (SOCO) benchmark evaluates structured object understanding in vision models through consistent part-level annotations and keypoint descriptions, revealing gaps between language-grounded localization and visual correspondence while demonstrating strong prediction of downstream task performance.
Measuring structured object understanding in vision foundation models remains challenging due to inconsistent evaluation protocols and limited part-level supervision. Semantic correspondence (SC) evaluates this capability by testing whether object parts can be matched across instances and categories under large variations in appearance, viewpoint, and geometry. To enable a systematic SC evaluation, we introduce SOCO, a new benchmark for Semantic Object Correspondence that introduces a taxonomy of correspondence types and provides consistent, functionally meaningful keypoint annotations across 100 categories and over 1M correspondence pairs. In addition, SOCO includes keypoint language descriptions, enabling the evaluation of large vision-language models (LVLMs) and their fine-grained part-level understanding. Comprehensive experiments reveal that (i) vision foundation backbones encode strong semantic structure but transfer correspondences poorly across related categories and only partially capture object-part position, (ii) LVLMs are stronger at text-prompted part localization than at visual-reference cross-image matching, exposing a gap between language-grounded localization and fine-grained visual correspondence, and (iii) correspondence performance predicts performance on dense downstream tasks, including segmentation, tracking, 3D pose estimation, and 3D detection, more strongly than ImageNet classification. Together, these findings position SOCO as a benchmark for structured, part-level representation quality in vision and multimodal foundation models.
Community
SOCO is a benchmark for evaluating structured, part-level understanding in vision and multimodal foundation models through semantic correspondence. It provides a taxonomy of correspondence types, functionally meaningful keypoint annotations across 100 categories, over 1M correspondence pairs, and language descriptions for evaluating LVLMs. Experiments show that current vision backbones encode semantic structure but struggle with cross-category correspondence and object-part position, while LVLMs perform better at text-prompted localization than visual-reference matching. SOCO also shows that semantic correspondence is a stronger predictor of dense downstream performance than ImageNet classification.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2605.31597 in a model README.md to link it from this page.
Cite arxiv.org/abs/2605.31597 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.