We revisit dataset distillation from an outcome-centric perspective. Rather than aligning process surrogates (per-step gradients or training trajectories), Influence Matching (Inf-Match) aligns the final outcome of training: it learns a compact synthetic set whose effect on the converged parameters matches that of the full dataset. Concretely, we introduce a fully differentiable, sample-level influence estimator that quantifies parameter shifts from adding or removing data, without time-consuming inverse-Hessian products or convexity assumptions. The estimator runs in linear time by unrolling the optimization dynamics and applying a first-order Taylor approximation. We then learn the synthetic set by minimizing the mismatch between its influence and that of the real dataset, yielding outcome alignment rather than heuristic process imitation. Inf-Match delivers the best accuracy across standard classification benchmarks. For instance, on Tiny-ImageNet (IPC=10), Inf-Match attains 31.5%, a +4.7% improvement over NCFM. Beyond classification, Inf-Match scales to vision-language distillation on Flickr30K, outperforming strong process-matching baselines. For instance, with 200 to 1000 synthetic samples, our method achieved a leading impressive average on image/text retrieval tasks, higher than NCFM by 2.5%.</p>\n","updatedAt":"2026-07-24T15:03:58.469Z","author":{"_id":"64dcdbd02d585b7d35f243a4","avatarUrl":"/avatars/a0f28727869c5f2913b3278a76bbf211.svg","fullname":"Haoru Tan","name":"thrshr","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8140043020248413},"editors":["thrshr"],"editorAvatarUrls":["/avatars/a0f28727869c5f2913b3278a76bbf211.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2607.16859","authors":[{"_id":"6a637e77b49b811f02241cea","name":"Haoru Tan","hidden":false},{"_id":"6a637e77b49b811f02241ceb","name":"Wang Wang","hidden":false},{"_id":"6a637e77b49b811f02241cec","name":"Sitong Wu","hidden":false},{"_id":"6a637e77b49b811f02241ced","name":"Xiuzhe Wu","hidden":false},{"_id":"6a637e77b49b811f02241cee","name":"Yangtian Sun","hidden":false},{"_id":"6a637e77b49b811f02241cef","name":"Chirui Chang","hidden":false},{"_id":"6a637e77b49b811f02241cf0","name":"Shaofeng Zhang","hidden":false},{"_id":"6a637e77b49b811f02241cf1","name":"Xiaojuan Qi","hidden":false}],"publishedAt":"2026-07-18T00:00:00.000Z","submittedOnDailyAt":"2026-07-24T00:00:00.000Z","title":"Dataset Distillation by Influence Matching","submittedOnDailyBy":{"_id":"64dcdbd02d585b7d35f243a4","avatarUrl":"/avatars/a0f28727869c5f2913b3278a76bbf211.svg","isPro":false,"fullname":"Haoru Tan","user":"thrshr","type":"user","name":"thrshr"},"summary":"We revisit dataset distillation from an outcome-centric perspective. Rather than aligning process surrogates (per-step gradients or training trajectories), Influence Matching (Inf-Match) aligns the final outcome of training: it learns a compact synthetic set whose effect on the converged parameters matches that of the full dataset. Concretely, we introduce a fully differentiable, sample-level influence estimator that quantifies parameter shifts from adding or removing data, without time-consuming inverse-Hessian products or convexity assumptions. The estimator runs in linear time by unrolling the optimization dynamics and applying a first-order Taylor approximation. We then learn the synthetic set by minimizing the mismatch between its influence and that of the real dataset, yielding outcome alignment rather than heuristic process imitation. Inf-Match delivers the best accuracy across standard classification benchmarks. For instance, on Tiny-ImageNet (IPC=10), Inf-Match attains 31.5\\%, a +4.7\\% improvement over NCFM. Beyond classification, Inf-Match scales to vision-language distillation on Flickr30K, outperforming strong process-matching baselines. For instance, with 200 to 1000 synthetic samples, our method achieved a leading impressive average on image/text retrieval tasks, higher than NCFM by 2.5\\%. The code will be released via https://github.com/hrtan/infmatch.","upvotes":0,"discussionId":"6a637e77b49b811f02241cf2","projectPage":"https://github.com/hrtan/infmatch","githubRepo":"https://github.com/hrtan/infmatch","githubRepoAddedBy":"user","githubStars":0,"organization":{"_id":"66deb312fd7d68a29348aa8d","name":"TheHKU","fullname":"Hong Kong University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/66dc525add44163a31059cf6/kyqlTADY27mPRTqznqQFL.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[],"acceptLanguages":["en"],"organization":{"_id":"66deb312fd7d68a29348aa8d","name":"TheHKU","fullname":"Hong Kong University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/66dc525add44163a31059cf6/kyqlTADY27mPRTqznqQFL.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2607/2607.16859.md","query":{}}">
Dataset Distillation by Influence Matching
Abstract
We revisit dataset distillation from an outcome-centric perspective. Rather than aligning process surrogates (per-step gradients or training trajectories), Influence Matching (Inf-Match) aligns the final outcome of training: it learns a compact synthetic set whose effect on the converged parameters matches that of the full dataset. Concretely, we introduce a fully differentiable, sample-level influence estimator that quantifies parameter shifts from adding or removing data, without time-consuming inverse-Hessian products or convexity assumptions. The estimator runs in linear time by unrolling the optimization dynamics and applying a first-order Taylor approximation. We then learn the synthetic set by minimizing the mismatch between its influence and that of the real dataset, yielding outcome alignment rather than heuristic process imitation. Inf-Match delivers the best accuracy across standard classification benchmarks. For instance, on Tiny-ImageNet (IPC=10), Inf-Match attains 31.5\%, a +4.7\% improvement over NCFM. Beyond classification, Inf-Match scales to vision-language distillation on Flickr30K, outperforming strong process-matching baselines. For instance, with 200 to 1000 synthetic samples, our method achieved a leading impressive average on image/text retrieval tasks, higher than NCFM by 2.5\%. The code will be released via https://github.com/hrtan/infmatch.
Community
We revisit dataset distillation from an outcome-centric perspective. Rather than aligning process surrogates (per-step gradients or training trajectories), Influence Matching (Inf-Match) aligns the final outcome of training: it learns a compact synthetic set whose effect on the converged parameters matches that of the full dataset. Concretely, we introduce a fully differentiable, sample-level influence estimator that quantifies parameter shifts from adding or removing data, without time-consuming inverse-Hessian products or convexity assumptions. The estimator runs in linear time by unrolling the optimization dynamics and applying a first-order Taylor approximation. We then learn the synthetic set by minimizing the mismatch between its influence and that of the real dataset, yielding outcome alignment rather than heuristic process imitation. Inf-Match delivers the best accuracy across standard classification benchmarks. For instance, on Tiny-ImageNet (IPC=10), Inf-Match attains 31.5%, a +4.7% improvement over NCFM. Beyond classification, Inf-Match scales to vision-language distillation on Flickr30K, outperforming strong process-matching baselines. For instance, with 200 to 1000 synthetic samples, our method achieved a leading impressive average on image/text retrieval tasks, higher than NCFM by 2.5%.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2607.16859 in a model README.md to link it from this page.
Cite arxiv.org/abs/2607.16859 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2607.16859 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.