Can we release all the weights of an LLM but still provide differential access to privileged users?</p>\n<p>Yes! We introduce: Tiered Language Models (TLMs). Define access tiers corresponding to different computation graphs over the same set of LLM parameters!</p>\n","updatedAt":"2026-06-23T13:21:02.527Z","author":{"_id":"631a523c04f8ed65eff16fb4","avatarUrl":"/avatars/2b284403c88f140d7bef283f729f7a3e.svg","fullname":"Arkil Patel","name":"arkilpatel","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8133106827735901},"editors":["arkilpatel"],"editorAvatarUrls":["/avatars/2b284403c88f140d7bef283f729f7a3e.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.21638","authors":[{"_id":"6a3a87b7fdcd3514343bb8ce","name":"Charbel El Feghali","hidden":false},{"_id":"6a3a87b7fdcd3514343bb8cf","name":"Arkil Patel","hidden":false},{"_id":"6a3a87b7fdcd3514343bb8d0","name":"Nicholas Meade","hidden":false},{"_id":"6a3a87b7fdcd3514343bb8d1","name":"Spandana Gella","hidden":false},{"_id":"6a3a87b7fdcd3514343bb8d2","name":"Verna Dankers","hidden":false},{"_id":"6a3a87b7fdcd3514343bb8d3","name":"Siva Reddy","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/631a523c04f8ed65eff16fb4/4tUsZxfG1x1kbYTvaM7xa.mp4"],"publishedAt":"2026-06-19T00:00:00.000Z","submittedOnDailyAt":"2026-06-23T00:00:00.000Z","title":"Toward Open Weight Models Without Risks: Separating Public and Private Capabilities in LLMs","submittedOnDailyBy":{"_id":"631a523c04f8ed65eff16fb4","avatarUrl":"/avatars/2b284403c88f140d7bef283f729f7a3e.svg","isPro":false,"fullname":"Arkil Patel","user":"arkilpatel","type":"user","name":"arkilpatel"},"summary":"Open-weight Large Language Models (LLMs) enable scientific progress and broad deployment. However, they make it difficult to control access to sensitive capabilities. Current practice either suppresses dangerous capabilities before release or mediates access through closed services that use specialized model variants, input/output monitors, and API permissions. The former is susceptible to jailbreaks while sacrificing capability for all users to mitigate the risks posed by a few, and the latter is fundamentally incompatible with open-weight release. In this paper, we propose Tiered Language Models (TLMs), where a single set of released weights supports multiple capability levels. In its default public configuration, a TLM behaves as a conventional LLM. A compact secret key specifies a permutation over a small parameter subset, inducing an alternative computation graph over the same weights that exposes additional capabilities. We develop a training protocol that jointly pretrains both configurations from scratch, then fine-tunes the keyed configuration on private data with regularization to preserve the public model's behavior. We pretrain 180M- and 650M-parameter TLMs and demonstrate that the keyed configuration can acquire a new language, gain instruction-following ability, and memorize private factual knowledge, whereas the public configuration exhibits none of these capabilities. Moreover, we show that our approach extends naturally to multiple hierarchical tiers. Because authorization operates on the model's weight structure rather than in the input space, the mechanism resists fine-tuning-based extraction and partial key compromise. In general, TLMs take a step toward reconciling open-weight release with selective capability control.","upvotes":7,"discussionId":"6a3a87b7fdcd3514343bb8d4","githubRepo":"https://github.com/McGill-NLP/tiered-language-models","githubRepoAddedBy":"user","ai_summary":"Tiered Language Models (TLMs) provide a framework for releasing large language models with configurable capability levels through secret keys that modify computation graphs while maintaining public model integrity.","ai_keywords":["Tiered Language Models","secret key","parameter subset","computation graph","pretraining","fine-tuning","regularization","instruction-following ability","private factual knowledge","authorization","weight structure"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","githubStars":1,"organization":{"_id":"63571619a482286f0dc9cce9","name":"MilaQuebec","fullname":"Mila – Quebec Artificial Intelligence Institute","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1666651643213-605238cb46a8a9249390a28e.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"631a523c04f8ed65eff16fb4","avatarUrl":"/avatars/2b284403c88f140d7bef283f729f7a3e.svg","isPro":false,"fullname":"Arkil Patel","user":"arkilpatel","type":"user"},{"_id":"6a2481eb67487d733e16bb70","avatarUrl":"/avatars/1d18fb03150f68dc9447760c24ccad76.svg","isPro":false,"fullname":"Atlas","user":"friday-and-co","type":"user"},{"_id":"6934b6677fd2c0224e6024f4","avatarUrl":"/avatars/f28f2be6e86ba3cee5b9fe7a5ebc57bf.svg","isPro":false,"fullname":"Charbel El Feghali","user":"charbelfeghalii","type":"user"},{"_id":"64527548fc4b47877aba7de0","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/64527548fc4b47877aba7de0/ht-mRRxNQT49A7NxArOGG.png","isPro":false,"fullname":"Nicholas Meade","user":"ncmeade","type":"user"},{"_id":"6525c70436f98a576a9f2c65","avatarUrl":"/avatars/c578c1d2d2c70a1af0c2ae28d815ed45.svg","isPro":false,"fullname":"Gaurav Kamath","user":"grvkamath","type":"user"},{"_id":"63f0546df1a47aaea5bcbae1","avatarUrl":"/avatars/9d708d88574cea2f17af37b659ef6a53.svg","isPro":false,"fullname":"Mehar Bhatia","user":"MeharBhatia","type":"user"},{"_id":"685be34822c268d54e056a8c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/qgE5WWcYzp32jbx9ewMht.png","isPro":false,"fullname":"Hoo","user":"owldacious","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"63571619a482286f0dc9cce9","name":"MilaQuebec","fullname":"Mila – Quebec Artificial Intelligence Institute","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1666651643213-605238cb46a8a9249390a28e.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.21638.md","query":{}}">
Toward Open Weight Models Without Risks: Separating Public and Private Capabilities in LLMs
Abstract
Tiered Language Models (TLMs) provide a framework for releasing large language models with configurable capability levels through secret keys that modify computation graphs while maintaining public model integrity.
Open-weight Large Language Models (LLMs) enable scientific progress and broad deployment. However, they make it difficult to control access to sensitive capabilities. Current practice either suppresses dangerous capabilities before release or mediates access through closed services that use specialized model variants, input/output monitors, and API permissions. The former is susceptible to jailbreaks while sacrificing capability for all users to mitigate the risks posed by a few, and the latter is fundamentally incompatible with open-weight release. In this paper, we propose Tiered Language Models (TLMs), where a single set of released weights supports multiple capability levels. In its default public configuration, a TLM behaves as a conventional LLM. A compact secret key specifies a permutation over a small parameter subset, inducing an alternative computation graph over the same weights that exposes additional capabilities. We develop a training protocol that jointly pretrains both configurations from scratch, then fine-tunes the keyed configuration on private data with regularization to preserve the public model's behavior. We pretrain 180M- and 650M-parameter TLMs and demonstrate that the keyed configuration can acquire a new language, gain instruction-following ability, and memorize private factual knowledge, whereas the public configuration exhibits none of these capabilities. Moreover, we show that our approach extends naturally to multiple hierarchical tiers. Because authorization operates on the model's weight structure rather than in the input space, the mechanism resists fine-tuning-based extraction and partial key compromise. In general, TLMs take a step toward reconciling open-weight release with selective capability control.
Community
Can we release all the weights of an LLM but still provide differential access to privileged users?
Yes! We introduce: Tiered Language Models (TLMs). Define access tiers corresponding to different computation graphs over the same set of LLM parameters!
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2606.21638 in a model README.md to link it from this page.
Cite arxiv.org/abs/2606.21638 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2606.21638 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.