Paper Link: <a href=\"https://arxiv.org/pdf/2608.12307\" rel=\"nofollow\">https://arxiv.org/pdf/2608.12307</a><br>A strong model can transfer capability to a weaker model at test time by automatically building an inference harness.</p>\n","updatedAt":"2026-08-13T02:34:10.846Z","author":{"_id":"665e121c6007027038fd4005","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/sIVBJAGM-Kneq9KMf8aXb.png","fullname":"Cheng Qian","name":"chengq9","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":8,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9227553009986877},"editors":["chengq9"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/sIVBJAGM-Kneq9KMf8aXb.png"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.12307","authors":[{"_id":"6a7d254b0ac8bee77474ee33","name":"Cheng Qian","hidden":false},{"_id":"6a7d254b0ac8bee77474ee34","name":"Wenting Zhao","hidden":false},{"_id":"6a7d254b0ac8bee77474ee35","name":"Liangwei Yang","hidden":false},{"_id":"6a7d254b0ac8bee77474ee36","name":"Heng Wang","hidden":false},{"_id":"6a7d254b0ac8bee77474ee37","name":"Jielin Qiu","hidden":false},{"_id":"6a7d254b0ac8bee77474ee38","name":"Heng Ji","hidden":false},{"_id":"6a7d254b0ac8bee77474ee39","name":"Silvio Savarese","hidden":false},{"_id":"6a7d254b0ac8bee77474ee3a","name":"Huan Wang","hidden":false},{"_id":"6a7d254b0ac8bee77474ee3b","name":"Shelby Heinecke","hidden":false}],"publishedAt":"2026-08-12T00:00:00.000Z","submittedOnDailyAt":"2026-08-13T00:00:00.000Z","title":"AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnesses","submittedOnDailyBy":{"_id":"665e121c6007027038fd4005","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/sIVBJAGM-Kneq9KMf8aXb.png","isPro":false,"fullname":"Cheng Qian","user":"chengq9","type":"user","name":"chengq9"},"summary":"Recent work on distillation transfers the capabilities of large models to smaller ones often by updating the latter's parameters, through teacher forcing, on-policy distillation, and related training-time methods. In this paper, we ask whether such transfer can instead occur at test time. We study strong-to-weak scaffolding: whether a stronger builder model can construct inference-time harnesses that help a weaker target model solve tasks more reliably without any parameter updates. Using four representative Theory-of-Mind benchmarks, each builder model uses 5% of the data as a validation set to iteratively refine its harness over multiple rounds, after which the finalized harness is evaluated on the full test set. Empirically, this form of test-time capability transfer is highly effective, nearly doubling average target-model performance from 0.49 to 0.91. Our analysis shows that the gains come primarily from offloading unstable model reasoning into deterministic code, benchmark-specific routing, and strict answer-format enforcement, rather than from encouraging the target model to reason more extensively or sample more broadly. We further find that builder-model reasoning effort improves harness quality monotonically, platform effects are modest relative to the builder model's own capability, and weaker target models receive the largest gains. These results suggest that inference-time harness design is an important complement to conventional training-time distillation, enabling strong models to transfer cognitive structure to weaker models without retraining.","upvotes":72,"discussionId":"6a7d254b0ac8bee77474ee3c","ai_summary":"Stronger models can build inference-time harnesses that substantially improve weaker models' task performance without parameter updates by offloading reasoning into structured code and routing.","ai_keywords":["strong-to-weak scaffolding","inference-time harnesses","Theory-of-Mind benchmarks","test-time capability transfer","deterministic code","benchmark-specific routing","answer-format enforcement","inference-time harness design","training-time distillation"],"ai_summary_model":"thinkingmachines/Inkling-Small","organization":{"_id":"65448bef5b5d9185ba3202b9","name":"UIUC-CS","fullname":"University of Illinois at Urbana-Champaign","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/65448b21fcb96b8b48733729/ycqcXFayMTTD_KpE37067.jpeg"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"665e121c6007027038fd4005","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/sIVBJAGM-Kneq9KMf8aXb.png","isPro":false,"fullname":"Cheng Qian","user":"chengq9","type":"user"},{"_id":"6449dbd8df4e6cb7eaef943e","avatarUrl":"/avatars/41a549a7b1cfe1d59ea16b3cbd2168cc.svg","isPro":false,"fullname":"ChengQ","user":"0Cheng0","type":"user"},{"_id":"684f8512aeaf14df648ce3f9","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/IF3Tf9_TNua0yJF7TPsm-.png","isPro":false,"fullname":"Cheng Qian","user":"chengq-sfr","type":"user"},{"_id":"6a7c16613d34d342bb461539","avatarUrl":"/avatars/5f74ed01471362a0c07defeacd10c5d1.svg","isPro":false,"fullname":"0Cheng1","user":"0Cheng1","type":"user"},{"_id":"6a6910e1611ac85b10b249d1","avatarUrl":"/avatars/fafa03d615903010d5a85bf25babf746.svg","isPro":false,"fullname":"quietlane","user":"quietlane","type":"user"},{"_id":"6a69119c30fefa25cc1194a6","avatarUrl":"/avatars/b5575f46b9c3029a08d80c80f39626c0.svg","isPro":false,"fullname":"silvernote","user":"silvernote","type":"user"},{"_id":"6a7c1502c9548a38dc9e3f3d","avatarUrl":"/avatars/7311ce04297c4d5ab112d568c81b91d9.svg","isPro":false,"fullname":"passionfire","user":"passionfire","type":"user"},{"_id":"6a7c12f88a355e774c4cbb27","avatarUrl":"/avatars/f513aeed3b38d2b32154038b7a0418bb.svg","isPro":false,"fullname":"happysnake","user":"happysnake","type":"user"},{"_id":"6a7c156d71fa900c3431026a","avatarUrl":"/avatars/0763cf27f0b3c7963e2c1c36c72f4b10.svg","isPro":false,"fullname":"ChengQ-Apodex","user":"ChengQ-Apodex","type":"user"},{"_id":"6a6915711275153b6eb73307","avatarUrl":"/avatars/f5d392b584f63da62b46b8dc7abf584c.svg","isPro":false,"fullname":"qcg3","user":"qcg3","type":"user"},{"_id":"64c5e944979493279b700cb2","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/vjFuPWw8Vl7b7gXB19Sk-.jpeg","isPro":false,"fullname":"Bingxiang He","user":"hbx","type":"user"},{"_id":"62fad90cc95d426ff8f4f2d5","avatarUrl":"/avatars/33f765c6a4b70bccab66c60e3f98145f.svg","isPro":false,"fullname":"Zhao","user":"inweriok","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":3,"organization":{"_id":"65448bef5b5d9185ba3202b9","name":"UIUC-CS","fullname":"University of Illinois at Urbana-Champaign","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/65448b21fcb96b8b48733729/ycqcXFayMTTD_KpE37067.jpeg"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.12307.md","query":{}}">
AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnesses
Abstract
Stronger models can build inference-time harnesses that substantially improve weaker models' task performance without parameter updates by offloading reasoning into structured code and routing.
Recent work on distillation transfers the capabilities of large models to smaller ones often by updating the latter's parameters, through teacher forcing, on-policy distillation, and related training-time methods. In this paper, we ask whether such transfer can instead occur at test time. We study strong-to-weak scaffolding: whether a stronger builder model can construct inference-time harnesses that help a weaker target model solve tasks more reliably without any parameter updates. Using four representative Theory-of-Mind benchmarks, each builder model uses 5% of the data as a validation set to iteratively refine its harness over multiple rounds, after which the finalized harness is evaluated on the full test set. Empirically, this form of test-time capability transfer is highly effective, nearly doubling average target-model performance from 0.49 to 0.91. Our analysis shows that the gains come primarily from offloading unstable model reasoning into deterministic code, benchmark-specific routing, and strict answer-format enforcement, rather than from encouraging the target model to reason more extensively or sample more broadly. We further find that builder-model reasoning effort improves harness quality monotonically, platform effects are modest relative to the builder model's own capability, and weaker target models receive the largest gains. These results suggest that inference-time harness design is an important complement to conventional training-time distillation, enabling strong models to transfer cognitive structure to weaker models without retraining.
Community
Paper Link: https://arxiv.org/pdf/2608.12307
A strong model can transfer capability to a weaker model at test time by automatically building an inference harness.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2608.12307 in a model README.md to link it from this page.
Cite arxiv.org/abs/2608.12307 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2608.12307 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.