Comparing Linear Probes with Mahalanobis Cosine Similarity</p>\n","updatedAt":"2026-06-23T19:35:14.486Z","author":{"_id":"6240c0c9d35be2c16ccfbe41","avatarUrl":"/avatars/d30d6c029a89939e7802f590480c5492.svg","fullname":"Josh Ying","name":"zfying","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7095214128494263},"editors":["zfying"],"editorAvatarUrls":["/avatars/d30d6c029a89939e7802f590480c5492.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.19603","authors":[{"_id":"6a36b4a0db23715e9da132c7","user":{"_id":"6240c0c9d35be2c16ccfbe41","avatarUrl":"/avatars/d30d6c029a89939e7802f590480c5492.svg","isPro":false,"fullname":"Josh Ying","user":"zfying","type":"user","name":"zfying"},"name":"Zhuofan Josh Ying","status":"claimed_verified","statusLastChangedAt":"2026-06-22T16:14:08.063Z","hidden":false},{"_id":"6a36b4a0db23715e9da132c8","name":"Peter Hase","hidden":false},{"_id":"6a36b4a0db23715e9da132c9","name":"Nikolaus Kriegeskorte","hidden":false}],"publishedAt":"2026-06-17T00:00:00.000Z","submittedOnDailyAt":"2026-06-23T00:00:00.000Z","title":"Comparing Linear Probes with Mahalanobis Cosine Similarity","submittedOnDailyBy":{"_id":"6240c0c9d35be2c16ccfbe41","avatarUrl":"/avatars/d30d6c029a89939e7802f590480c5492.svg","isPro":false,"fullname":"Josh Ying","user":"zfying","type":"user","name":"zfying"},"summary":"Linear probes are widely used in interpretability research and often compared by cosine similarity. The Mahalanobis cosine similarity (MCS) between two directions, which reweights the inner product by test data covariance, is a natural task-aware refinement. Ying et al. (2026) report that a probe's MCS to a reference probe trained on the out-of-distribution (OOD) data near-perfectly linearly predicts the probe's OOD AUROC (R^2 = 0.98). Here, we extend this empirical finding across models, layers, and concept domains, and prove this general phenomenon in closed form: For balanced classes whose projections are Gaussian, OOD AUROC and MCS to the reference probe are linear because both are sigmoid-shaped functions of the probe's signal-to-noise ratio (SNR) on the test data. The theory also predicts when this linearity fails, which we verify empirically. MCS offers a theoretically grounded and empirically effective alternative to Euclidean cosine similarity for comparing linear probes.","upvotes":3,"discussionId":"6a36b4a1db23715e9da132ca","ai_summary":"The Mahalanobis cosine similarity provides a theoretically grounded method for comparing linear probes that correlates strongly with out-of-distribution performance metrics.","ai_keywords":["Mahalanobis cosine similarity","linear probes","out-of-distribution AUROC","signal-to-noise ratio","Gaussian projections","sigmoid-shaped functions","test data covariance","reference probe"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct"},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6240c0c9d35be2c16ccfbe41","avatarUrl":"/avatars/d30d6c029a89939e7802f590480c5492.svg","isPro":false,"fullname":"Josh Ying","user":"zfying","type":"user"},{"_id":"697c8b15a7f796854ef333c4","avatarUrl":"/avatars/94de3a736fac914944f1b57609e3819a.svg","isPro":false,"fullname":"Joel Wang","user":"joelhenwang","type":"user"},{"_id":"69ffa09cf204ce9d1821803a","avatarUrl":"/avatars/caca3cc632dc36ba8babc753d81cb4fd.svg","isPro":false,"fullname":"Peter Anthony Palmtree","user":"peteranthonypalmtree","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.19603.md","query":{}}">
Comparing Linear Probes with Mahalanobis Cosine Similarity
Abstract
The Mahalanobis cosine similarity provides a theoretically grounded method for comparing linear probes that correlates strongly with out-of-distribution performance metrics.
Linear probes are widely used in interpretability research and often compared by cosine similarity. The Mahalanobis cosine similarity (MCS) between two directions, which reweights the inner product by test data covariance, is a natural task-aware refinement. Ying et al. (2026) report that a probe's MCS to a reference probe trained on the out-of-distribution (OOD) data near-perfectly linearly predicts the probe's OOD AUROC (R^2 = 0.98). Here, we extend this empirical finding across models, layers, and concept domains, and prove this general phenomenon in closed form: For balanced classes whose projections are Gaussian, OOD AUROC and MCS to the reference probe are linear because both are sigmoid-shaped functions of the probe's signal-to-noise ratio (SNR) on the test data. The theory also predicts when this linearity fails, which we verify empirically. MCS offers a theoretically grounded and empirically effective alternative to Euclidean cosine similarity for comparing linear probes.
Community
Comparing Linear Probes with Mahalanobis Cosine Similarity
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2606.19603 in a model README.md to link it from this page.
Cite arxiv.org/abs/2606.19603 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2606.19603 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.