Accepted to TPAMI2026. Extension of the CVPR2024 version (arXiv:2405.02581).</p>\n","updatedAt":"2026-06-12T12:20:56.035Z","author":{"_id":"669edb04cd29e1b976a64c73","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/669edb04cd29e1b976a64c73/coTTDl4pBCNOr8sXvIkkr.jpeg","fullname":"Niccolò Biondi","name":"NiccoBiondi","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":6,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.47923630475997925},"editors":["NiccoBiondi"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/669edb04cd29e1b976a64c73/coTTDl4pBCNOr8sXvIkkr.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.12488","authors":[{"_id":"6a2bf925a0d4daae4285eec2","name":"Niccolò Biondi","hidden":false},{"_id":"6a2bf925a0d4daae4285eec3","name":"Federico Pernici","hidden":false},{"_id":"6a2bf925a0d4daae4285eec4","name":"Simone Ricci","hidden":false},{"_id":"6a2bf925a0d4daae4285eec5","name":"Alberto Del Bimbo","hidden":false}],"publishedAt":"2026-06-10T00:00:00.000Z","submittedOnDailyAt":"2026-06-12T00:00:00.000Z","title":"A Stationary (and Therefore Compatible) Representation is All You Need","submittedOnDailyBy":{"_id":"669edb04cd29e1b976a64c73","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/669edb04cd29e1b976a64c73/coTTDl4pBCNOr8sXvIkkr.jpeg","isPro":false,"fullname":"Niccolò Biondi","user":"NiccoBiondi","type":"user","name":"NiccoBiondi"},"summary":"Learning compatible representations aims to learn feature representations that can be used interchangeably over time whenever a model undergoes updates. In this paper, we demonstrate that stationary representations learned by d-Simplex fixed classifiers imply compatibility as in its formal definition. This result establishes a foundation for future works and can be directly exploited in practical learning scenarios. We address the challenge of learning compatibility using d-Simplex fixed classifiers when the model is sequentially fine-tuned. Learning according to a d-Simplex fixed classifier with the cross-entropy loss aligns feature distributions at the first-order statistics. Consequently, it may not fully capture higher-order dependencies in the representation between model updates. To address this issue, we demonstrate that training the model using a d-Simplex fixed classifier through a convex combination of the cross-entropy loss and a contrastive loss not only captures higher-order dependencies, but is also equivalent to learning with the cross-entropy under the compatibility constraints. We confirm our findings with extensive experiments also considering a new scenario where a pre-trained model is sequentially fine-tuned and occasionally replaced with an improved model. We show that stationary representations enable uninterrupted retrieval services (without reprocessing gallery images) while improving performance during model updates and replacements, achieving state-of-the-art. Code at https://github.com/miccunifi/iamcl2r.","upvotes":0,"discussionId":"6a2bf926a0d4daae4285eec6","projectPage":"https://niccobiondi.github.io/projects/stationary-tpami/","ai_summary":"Stationary representations learned through d-Simplex fixed classifiers ensure model compatibility during sequential fine-tuning and updates, enabling continuous retrieval services without reprocessing.","ai_keywords":["d-Simplex fixed classifiers","cross-entropy loss","contrastive loss","feature representations","model compatibility","sequential fine-tuning","stationary representations","higher-order dependencies","convex combination"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","organization":{"_id":"647ef9ce45baf21ad707cc59","name":"MHUGLab","fullname":"Multimedia and Human Understanding Group","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/647ef81a45baf21ad707bfb1/VfcRM08QNOTXAqIUcf-WO.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[],"acceptLanguages":["en"],"organization":{"_id":"647ef9ce45baf21ad707cc59","name":"MHUGLab","fullname":"Multimedia and Human Understanding Group","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/647ef81a45baf21ad707bfb1/VfcRM08QNOTXAqIUcf-WO.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.12488.md","query":{}}">
A Stationary (and Therefore Compatible) Representation is All You Need
Abstract
Stationary representations learned through d-Simplex fixed classifiers ensure model compatibility during sequential fine-tuning and updates, enabling continuous retrieval services without reprocessing.
Learning compatible representations aims to learn feature representations that can be used interchangeably over time whenever a model undergoes updates. In this paper, we demonstrate that stationary representations learned by d-Simplex fixed classifiers imply compatibility as in its formal definition. This result establishes a foundation for future works and can be directly exploited in practical learning scenarios. We address the challenge of learning compatibility using d-Simplex fixed classifiers when the model is sequentially fine-tuned. Learning according to a d-Simplex fixed classifier with the cross-entropy loss aligns feature distributions at the first-order statistics. Consequently, it may not fully capture higher-order dependencies in the representation between model updates. To address this issue, we demonstrate that training the model using a d-Simplex fixed classifier through a convex combination of the cross-entropy loss and a contrastive loss not only captures higher-order dependencies, but is also equivalent to learning with the cross-entropy under the compatibility constraints. We confirm our findings with extensive experiments also considering a new scenario where a pre-trained model is sequentially fine-tuned and occasionally replaced with an improved model. We show that stationary representations enable uninterrupted retrieval services (without reprocessing gallery images) while improving performance during model updates and replacements, achieving state-of-the-art. Code at https://github.com/miccunifi/iamcl2r.
Community
Accepted to TPAMI2026. Extension of the CVPR2024 version (arXiv:2405.02581).
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2606.12488 in a model README.md to link it from this page.
Cite arxiv.org/abs/2606.12488 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2606.12488 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.