We propose Verification-Aware Training (VAT), a plug-in framework that improves speculative decoding by training the draft model using simulated verification patterns and adaptive weighting.</p>\n","updatedAt":"2026-09-01T06:20:58.140Z","author":{"_id":"61839579a658bd70108e88c0","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/61839579a658bd70108e88c0/IKyBDiFi4qH1Qy3nJkW-R.jpeg","fullname":"Geonmo Gu","name":"Geonmo","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":14,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8836564421653748},"editors":["Geonmo"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/61839579a658bd70108e88c0/IKyBDiFi4qH1Qy3nJkW-R.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.30135","authors":[{"_id":"6a9644bdcd6ebc484732eccd","name":"Geonmo Gu","hidden":false},{"_id":"6a9644bdcd6ebc484732ecce","name":"Byeongho Heo","hidden":false},{"_id":"6a9644bdcd6ebc484732eccf","name":"HeeJae Jun","hidden":false},{"_id":"6a9644bdcd6ebc484732ecd0","name":"Yoohoon Kang","hidden":false},{"_id":"6a9644bdcd6ebc484732ecd1","name":"Sangmin Lee","hidden":false},{"_id":"6a9644bdcd6ebc484732ecd2","name":"Sangdoo Yun","hidden":false},{"_id":"6a9644bdcd6ebc484732ecd3","name":"Dongyoon Han","hidden":false}],"publishedAt":"2026-08-31T00:00:00.000Z","submittedOnDailyAt":"2026-09-01T00:00:00.000Z","title":"Verification-Aware Training for Speculative Decoding","submittedOnDailyBy":{"_id":"61839579a658bd70108e88c0","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/61839579a658bd70108e88c0/IKyBDiFi4qH1Qy3nJkW-R.jpeg","isPro":false,"fullname":"Geonmo Gu","user":"Geonmo","type":"user","name":"Geonmo"},"summary":"Speculative decoding accelerates large language model inference by using a draft model to generate candidate tokens, which are verified by the target model in a single forward pass. Verification proceeds sequentially and discards every position from the first rejection onward, yet existing draft training relies on token-level imitation of the target with a fixed per-position weighting that reflects neither property. We introduce Verification-Aware Training (VAT), a plug-in framework that simulates verification at every training step and turns the resulting accept and reject patterns into supervision. VAT consists of two components: (i) a verification head, a lightweight jointly trained binary classifier that supervises the draft model on whether each position survives sequential verification; (ii) verification-adaptive weighting, which replaces the fixed weighting schedule by keeping full weight up to each sample's first rejection point and re-anchoring the decay to start there. VAT modifies only the training objective, so it can be layered on top of existing methods without changing the draft architecture, the target model, or the inference procedure. Applied to EAGLE-3 and DFlash on Qwen3-4B, Qwen3-8B, and LLaMA-3.1-8B, VAT improves average acceptance length by up to 11.4% and wall-clock speedup by up to 8.7%, with consistent gains across math, code, and chat benchmarks. Code will be available at https://github.com/naver-ai/vat","upvotes":2,"discussionId":"6a9644bdcd6ebc484732ecd4","ai_summary":"Verification-Aware Training improves draft models for speculative decoding by simulating sequential verification during training and adapting loss weights to acceptance patterns.","ai_keywords":["speculative decoding","draft model","verification","Verification-Aware Training","verification head","verification-adaptive weighting","EAGLE-3","DFlash"],"ai_summary_model":"thinkingmachines/Inkling-Small","organization":{"_id":"64ffe603efd273eec7768bde","name":"naver-ai","fullname":"NAVER AI Lab","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/64ff1755b75685dd7a46e146/Zj2bxgq31oSqwVrw16IE_.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"61839579a658bd70108e88c0","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/61839579a658bd70108e88c0/IKyBDiFi4qH1Qy3nJkW-R.jpeg","isPro":false,"fullname":"Geonmo Gu","user":"Geonmo","type":"user"},{"_id":"63ac5701c21e60a3e9b58aa7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63ac5701c21e60a3e9b58aa7/g6EX7diOpuA94R2ab-rZC.png","isPro":true,"fullname":"Dipankar Sarkar","user":"dipankarsarkar","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"64ffe603efd273eec7768bde","name":"naver-ai","fullname":"NAVER AI Lab","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/64ff1755b75685dd7a46e146/Zj2bxgq31oSqwVrw16IE_.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.30135.md","query":{}}">
Verification-Aware Training for Speculative Decoding
Abstract
Verification-Aware Training improves draft models for speculative decoding by simulating sequential verification during training and adapting loss weights to acceptance patterns.
Speculative decoding accelerates large language model inference by using a draft model to generate candidate tokens, which are verified by the target model in a single forward pass. Verification proceeds sequentially and discards every position from the first rejection onward, yet existing draft training relies on token-level imitation of the target with a fixed per-position weighting that reflects neither property. We introduce Verification-Aware Training (VAT), a plug-in framework that simulates verification at every training step and turns the resulting accept and reject patterns into supervision. VAT consists of two components: (i) a verification head, a lightweight jointly trained binary classifier that supervises the draft model on whether each position survives sequential verification; (ii) verification-adaptive weighting, which replaces the fixed weighting schedule by keeping full weight up to each sample's first rejection point and re-anchoring the decay to start there. VAT modifies only the training objective, so it can be layered on top of existing methods without changing the draft architecture, the target model, or the inference procedure. Applied to EAGLE-3 and DFlash on Qwen3-4B, Qwen3-8B, and LLaMA-3.1-8B, VAT improves average acceptance length by up to 11.4% and wall-clock speedup by up to 8.7%, with consistent gains across math, code, and chat benchmarks. Code will be available at https://github.com/naver-ai/vat
Community
We propose Verification-Aware Training (VAT), a plug-in framework that improves speculative decoding by training the draft model using simulated verification patterns and adaptive weighting.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2608.30135 in a model README.md to link it from this page.
Cite arxiv.org/abs/2608.30135 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2608.30135 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.