Reinforcement fine-tuning (RFT) is increasingly used to strengthen the reasoning abilities of large models, yet its effectiveness is bound by how training data are selected and used. Most data-centric RFT methods rely on static or heuristic sample selection, implicitly assuming a sample's value is fixed over training. This overlooks the non-stationary dynamics of policy learning and can lead to suboptimal updates. We propose Dynamic Important Example Mining (DIEM), a principled and fully automated framework that makes data utilization adaptive throughout RFT. DIEM integrates two components into each optimization step: (i) a gradient-alignment importance estimator that efficiently approximates each sample's marginal contribution to policy improvement; and (ii) a constrained batch reweighting scheme that maximizes aggregate utility while preserving the update's gradient magnitude to stabilize optimization. Across several reasoning benchmarks, DIEM consistently outperforms strong static and dynamic baselines.</p>\n","updatedAt":"2026-09-01T04:52:45.702Z","author":{"_id":"64dcdbd02d585b7d35f243a4","avatarUrl":"/avatars/a0f28727869c5f2913b3278a76bbf211.svg","fullname":"Haoru Tan","name":"thrshr","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8953512907028198},"editors":["thrshr"],"editorAvatarUrls":["/avatars/a0f28727869c5f2913b3278a76bbf211.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.29252","authors":[{"_id":"6a9659d4cd6ebc484732edfa","name":"Haoru Tan","hidden":false},{"_id":"6a9659d4cd6ebc484732edfb","name":"Sitong Wu","hidden":false},{"_id":"6a9659d4cd6ebc484732edfc","name":"Yanfeng Chen","hidden":false},{"_id":"6a9659d4cd6ebc484732edfd","name":"Shizhen Zhao","hidden":false},{"_id":"6a9659d4cd6ebc484732edfe","name":"Yang-Tian Sun","hidden":false},{"_id":"6a9659d4cd6ebc484732edff","name":"Tianjia Liu","hidden":false},{"_id":"6a9659d4cd6ebc484732ee00","name":"Chirui Chang","hidden":false},{"_id":"6a9659d4cd6ebc484732ee01","name":"Shaofeng Zhang","hidden":false},{"_id":"6a9659d4cd6ebc484732ee02","name":"Samm Sun","hidden":false},{"_id":"6a9659d4cd6ebc484732ee03","name":"Xiuzhe Wu","hidden":false},{"_id":"6a9659d4cd6ebc484732ee04","name":"Ruobing Xie","hidden":false},{"_id":"6a9659d4cd6ebc484732ee05","name":"Xiaojuan Qi","hidden":false}],"publishedAt":"2026-08-29T00:00:00.000Z","submittedOnDailyAt":"2026-09-01T00:00:00.000Z","title":"Dynamic Important Example Mining for Reinforcement Finetuning","submittedOnDailyBy":{"_id":"64dcdbd02d585b7d35f243a4","avatarUrl":"/avatars/a0f28727869c5f2913b3278a76bbf211.svg","isPro":false,"fullname":"Haoru Tan","user":"thrshr","type":"user","name":"thrshr"},"summary":"Reinforcement fine-tuning (RFT) is increasingly used to strengthen the reasoning abilities of large models, yet its effectiveness is bound by how training data are selected and used. Most data-centric RFT methods rely on static or heuristic sample selection, implicitly assuming a sample's value is fixed over training. This overlooks the non-stationary dynamics of policy learning and can lead to suboptimal updates. We propose Dynamic Important Example Mining (DIEM), a principled and fully automated framework that makes data utilization adaptive throughout RFT. DIEM integrates two components into each optimization step: (i) a gradient-alignment importance estimator that efficiently approximates each sample's marginal contribution to policy improvement; and (ii) a constrained batch reweighting scheme that maximizes aggregate utility while preserving the update's gradient magnitude to stabilize optimization. Across several reasoning benchmarks, DIEM consistently outperforms strong static and dynamic baselines. The code will be released via https://github.com/hrtan/DIEM.","upvotes":0,"discussionId":"6a9659d4cd6ebc484732ee06","githubRepo":"https://github.com/hrtan/DIEM","githubRepoAddedBy":"user","ai_summary":"DIEM adaptively selects and reweights training examples during reinforcement fine-tuning by estimating each sample's marginal contribution to policy improvement and constraining batch updates to stabilize optimization.","ai_keywords":["reinforcement fine-tuning","gradient-alignment importance estimator","constrained batch reweighting","policy improvement","non-stationary dynamics"],"ai_summary_model":"thinkingmachines/Inkling-Small","githubStars":0,"organization":{"_id":"6645f953c39288df638dbdd5","name":"Tencent-Hunyuan","fullname":"Tencent Hunyuan","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/62d22496c58f969c152bcefd/woKSjt2wXvBNKussyYPsa.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[],"acceptLanguages":["en"],"organization":{"_id":"6645f953c39288df638dbdd5","name":"Tencent-Hunyuan","fullname":"Tencent Hunyuan","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/62d22496c58f969c152bcefd/woKSjt2wXvBNKussyYPsa.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.29252.md","query":{}}">
Dynamic Important Example Mining for Reinforcement Finetuning
Abstract
DIEM adaptively selects and reweights training examples during reinforcement fine-tuning by estimating each sample's marginal contribution to policy improvement and constraining batch updates to stabilize optimization.
Reinforcement fine-tuning (RFT) is increasingly used to strengthen the reasoning abilities of large models, yet its effectiveness is bound by how training data are selected and used. Most data-centric RFT methods rely on static or heuristic sample selection, implicitly assuming a sample's value is fixed over training. This overlooks the non-stationary dynamics of policy learning and can lead to suboptimal updates. We propose Dynamic Important Example Mining (DIEM), a principled and fully automated framework that makes data utilization adaptive throughout RFT. DIEM integrates two components into each optimization step: (i) a gradient-alignment importance estimator that efficiently approximates each sample's marginal contribution to policy improvement; and (ii) a constrained batch reweighting scheme that maximizes aggregate utility while preserving the update's gradient magnitude to stabilize optimization. Across several reasoning benchmarks, DIEM consistently outperforms strong static and dynamic baselines. The code will be released via https://github.com/hrtan/DIEM.
Community
Reinforcement fine-tuning (RFT) is increasingly used to strengthen the reasoning abilities of large models, yet its effectiveness is bound by how training data are selected and used. Most data-centric RFT methods rely on static or heuristic sample selection, implicitly assuming a sample's value is fixed over training. This overlooks the non-stationary dynamics of policy learning and can lead to suboptimal updates. We propose Dynamic Important Example Mining (DIEM), a principled and fully automated framework that makes data utilization adaptive throughout RFT. DIEM integrates two components into each optimization step: (i) a gradient-alignment importance estimator that efficiently approximates each sample's marginal contribution to policy improvement; and (ii) a constrained batch reweighting scheme that maximizes aggregate utility while preserving the update's gradient magnitude to stabilize optimization. Across several reasoning benchmarks, DIEM consistently outperforms strong static and dynamic baselines.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2608.29252 in a model README.md to link it from this page.
Cite arxiv.org/abs/2608.29252 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2608.29252 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.