The main bottleneck in contrastive authorship attribution is not whether stylistic information exists in the encoder, but whether the scoring mechanism can preserve and exploit it.","html":"<blockquote>\n<p>The main bottleneck in contrastive authorship attribution is not whether stylistic information exists in the encoder, but whether the scoring mechanism can preserve and exploit it.</p>\n</blockquote>\n","updatedAt":"2026-05-20T09:02:44.576Z","author":{"_id":"622a058138f0b01c1c2b33c9","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/622a058138f0b01c1c2b33c9/fZ2T_BJU9gbXGuxgbZ_OI.jpeg","fullname":"Francis Kulumba","name":"Madjakul","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8125118017196655},"editors":["Madjakul"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/622a058138f0b01c1c2b33c9/fZ2T_BJU9gbXGuxgbZ_OI.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2605.19908","authors":[{"_id":"6a0d77400cc88a0d483d3737","name":"Francis Kulumba","hidden":false},{"_id":"6a0d77400cc88a0d483d3738","name":"Guillaume Vimont","hidden":false},{"_id":"6a0d77400cc88a0d483d3739","name":"Laurent Romary","hidden":false},{"_id":"6a0d77400cc88a0d483d373a","name":"Florian Cafiero","hidden":false}],"publishedAt":"2026-05-19T00:00:00.000Z","submittedOnDailyAt":"2026-05-20T00:00:00.000Z","title":"Where Does Authorship Signal Emerge in Encoder-Based Language Models?","submittedOnDailyBy":{"_id":"622a058138f0b01c1c2b33c9","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/622a058138f0b01c1c2b33c9/fZ2T_BJU9gbXGuxgbZ_OI.jpeg","isPro":false,"fullname":"Francis Kulumba","user":"Madjakul","type":"user","name":"Madjakul"},"summary":"Authorship attribution models fine-tuned with the same pretrained encoder, data, and loss can differ four-fold in performance depending only on their scoring mechanism. We use mechanistic interpretability tools to explain this gap. Stylistic features such as word length, punctuation density, and function-word frequency are equally available at every layer in every model, including in an off-the-shelf control encoder, hence the gap not coming from representation quality. Instead, causal intervention shows that the scorer determines where the encoder consolidates authorship signal. Mean pooling forces consolidation by early to mid layers, while late interaction defers it to later layers. We further derive this difference from the gradient structure of each scorer, and training dynamics reveal distinct learning trajectories that follow from that difference.","upvotes":2,"discussionId":"6a0d77410cc88a0d483d373b","githubRepo":"https://github.com/Madjakul/DeepStylometry","githubRepoAddedBy":"user","ai_summary":"Authorship attribution model performance varies significantly based on scoring mechanisms rather than representation quality, with different consolidation layers of authorship signals determined by gradient structures and training dynamics.","ai_keywords":["authorship attribution","pretrained encoder","scoring mechanism","mechanistic interpretability","stylistic features","gradient structure","training dynamics"],"githubStars":1,"organization":{"_id":"602ba30dc4f8038e9a1e0a60","name":"almanach","fullname":"ALMAnaCH (Inria)","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1613472488646-602ba2a739515f8d31237967.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"661ab1f1fa3b144a381fa454","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/661ab1f1fa3b144a381fa454/IlpZBb9NCjo7ntFwMIH53.png","isPro":true,"fullname":"Urro","user":"urroxyz","type":"user"},{"_id":"622a058138f0b01c1c2b33c9","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/622a058138f0b01c1c2b33c9/fZ2T_BJU9gbXGuxgbZ_OI.jpeg","isPro":false,"fullname":"Francis Kulumba","user":"Madjakul","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"602ba30dc4f8038e9a1e0a60","name":"almanach","fullname":"ALMAnaCH (Inria)","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1613472488646-602ba2a739515f8d31237967.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2605/2605.19908.md"}">
Where Does Authorship Signal Emerge in Encoder-Based Language Models?
Abstract
Authorship attribution model performance varies significantly based on scoring mechanisms rather than representation quality, with different consolidation layers of authorship signals determined by gradient structures and training dynamics.
AI-generated summary
Authorship attribution models fine-tuned with the same pretrained encoder, data, and loss can differ four-fold in performance depending only on their scoring mechanism. We use mechanistic interpretability tools to explain this gap. Stylistic features such as word length, punctuation density, and function-word frequency are equally available at every layer in every model, including in an off-the-shelf control encoder, hence the gap not coming from representation quality. Instead, causal intervention shows that the scorer determines where the encoder consolidates authorship signal. Mean pooling forces consolidation by early to mid layers, while late interaction defers it to later layers. We further derive this difference from the gradient structure of each scorer, and training dynamics reveal distinct learning trajectories that follow from that difference.
Community
The main bottleneck in contrastive authorship attribution is not whether stylistic information exists in the encoder, but whether the scoring mechanism can preserve and exploit it.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2605.19908 in a model README.md to link it from this page.
Cite arxiv.org/abs/2605.19908 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2605.19908 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.