TLDR: We introduce Explorative Modeling, a new paradigm for generative modeling that acts as a third pretraining axis when added to existing generative models, and also enables end-to-end generation. Increasing exploration monotonically improves existing models across images, video, and language, and the gains grow with scale (7%→36% with data, 13%→23% with parameters). Concretely, Explorative Models (XMs) reach 6.2× sample efficiency, 4.1× FLOP efficiency, and 47% better parameter efficiency. Exploration also enables scaling generalization, and even scaling how end-to-end existing models are. As end-to-end generative models, XMs match diffusion on control tasks with up to 256× less inference compute.<br><a href=\"https://cdn-uploads.huggingface.co/production/uploads/63fbbff41b4b1bd4e7042fd1/OTyubrPu68b3MEtGTh58f.png\" rel=\"nofollow\"><img src=\"https://cdn-uploads.huggingface.co/production/uploads/63fbbff41b4b1bd4e7042fd1/OTyubrPu68b3MEtGTh58f.png\" alt=\"banner_xm\"></a></p>\n","updatedAt":"2026-07-31T03:57:33.535Z","author":{"_id":"63fbbff41b4b1bd4e7042fd1","avatarUrl":"/avatars/d186d0859046b35723fe1156d8f68917.svg","fullname":"Alexi Gladstone","name":"alexiglad","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":8,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8097206354141235},"editors":["alexiglad"],"editorAvatarUrls":["/avatars/d186d0859046b35723fe1156d8f68917.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2607.27372","authors":[{"_id":"6a6c1c64202e2d9e3ffdb7ba","user":{"_id":"63fbbff41b4b1bd4e7042fd1","avatarUrl":"/avatars/d186d0859046b35723fe1156d8f68917.svg","isPro":false,"fullname":"Alexi Gladstone","user":"alexiglad","type":"user","name":"alexiglad"},"name":"Alexi Gladstone","status":"claimed_verified","statusLastChangedAt":"2026-07-31T08:45:05.628Z","hidden":false},{"_id":"6a6c1c64202e2d9e3ffdb7bb","name":"Heng Ji","hidden":false},{"_id":"6a6c1c64202e2d9e3ffdb7bc","name":"Yilun Du","hidden":false}],"publishedAt":"2026-07-29T00:00:00.000Z","submittedOnDailyAt":"2026-07-31T00:00:00.000Z","title":"Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation","submittedOnDailyBy":{"_id":"63fbbff41b4b1bd4e7042fd1","avatarUrl":"/avatars/d186d0859046b35723fe1156d8f68917.svg","isPro":false,"fullname":"Alexi Gladstone","user":"alexiglad","type":"user","name":"alexiglad"},"summary":"The deep learning revolution, kicked off by AlexNet, taught us that end-to-end training beats decomposing a problem into hand-designed stages. Generative modeling, however, has remained the exception-despite generative models being remarkably capable, they are still not trained end-to-end. This is because, at its core, generative modeling is about handling distributions with many modes, and existing scalable approaches handle this the same way, by factoring the generation procedure, which prevents end-to-end generation. In this work, we introduce Explorative Modeling, a new paradigm that instead factors the training loop, exploring K candidate matches between model generations and data, and training on the best, so predictions commit to modes rather than blurring them. We find Explorative Models (XMs) useful in two settings. First, increasing exploration adds a third pretraining axis beyond parameters and data for existing generative models-where scaling exploration monotonically improves performance across both continuous and discrete domains (images, video, and language). Notably, gains from exploration increase with scale, climbing from 7% to 36% as data scales and from 13% to 23% as models grow, with efficiency gains more than doubling at 3x the compute. Concretely, exploration improves FLOP efficiency by 4.1x, sample efficiency by 6.2x, parameter efficiency by 47%, lifts the strongest of image-generation recipes to a near-state-of-the-art 1.43 FID on ImageNet without guidance, enables scaling how end-to-end existing models are, and unlocks scaling generalization. Second, XMs enable end-to-end reconstructive generative modeling, matching diffusion on control tasks with 16-256x fewer inference steps. Together, these results establish XMs as both a new pretraining axis for existing generative models and a standalone end-to-end generative modeling paradigm.","upvotes":2,"discussionId":"6a6c1c64202e2d9e3ffdb7bd","projectPage":"https://explorative-modeling.github.io/","githubRepo":"https://github.com/alexiglad/XM","githubRepoAddedBy":"user","githubStars":2,"organization":{"_id":"60212a089f64108326fac7c2","name":"illinois","fullname":"University of Illinois at Urbana-Champaign","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1612786274096-6021121cfb1b47827d667074.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"63fbbff41b4b1bd4e7042fd1","avatarUrl":"/avatars/d186d0859046b35723fe1156d8f68917.svg","isPro":false,"fullname":"Alexi Gladstone","user":"alexiglad","type":"user"},{"_id":"6a04677f705f625ac6bb170e","avatarUrl":"/avatars/a16404b988eb7a7992cac9dbdd1c6f04.svg","isPro":false,"fullname":"Longcang Wang","user":"LongcangWang","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"60212a089f64108326fac7c2","name":"illinois","fullname":"University of Illinois at Urbana-Champaign","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1612786274096-6021121cfb1b47827d667074.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2607/2607.27372.md","query":{}}">
Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation
Abstract
The deep learning revolution, kicked off by AlexNet, taught us that end-to-end training beats decomposing a problem into hand-designed stages. Generative modeling, however, has remained the exception-despite generative models being remarkably capable, they are still not trained end-to-end. This is because, at its core, generative modeling is about handling distributions with many modes, and existing scalable approaches handle this the same way, by factoring the generation procedure, which prevents end-to-end generation. In this work, we introduce Explorative Modeling, a new paradigm that instead factors the training loop, exploring K candidate matches between model generations and data, and training on the best, so predictions commit to modes rather than blurring them. We find Explorative Models (XMs) useful in two settings. First, increasing exploration adds a third pretraining axis beyond parameters and data for existing generative models-where scaling exploration monotonically improves performance across both continuous and discrete domains (images, video, and language). Notably, gains from exploration increase with scale, climbing from 7% to 36% as data scales and from 13% to 23% as models grow, with efficiency gains more than doubling at 3x the compute. Concretely, exploration improves FLOP efficiency by 4.1x, sample efficiency by 6.2x, parameter efficiency by 47%, lifts the strongest of image-generation recipes to a near-state-of-the-art 1.43 FID on ImageNet without guidance, enables scaling how end-to-end existing models are, and unlocks scaling generalization. Second, XMs enable end-to-end reconstructive generative modeling, matching diffusion on control tasks with 16-256x fewer inference steps. Together, these results establish XMs as both a new pretraining axis for existing generative models and a standalone end-to-end generative modeling paradigm.
Community
TLDR: We introduce Explorative Modeling, a new paradigm for generative modeling that acts as a third pretraining axis when added to existing generative models, and also enables end-to-end generation. Increasing exploration monotonically improves existing models across images, video, and language, and the gains grow with scale (7%→36% with data, 13%→23% with parameters). Concretely, Explorative Models (XMs) reach 6.2× sample efficiency, 4.1× FLOP efficiency, and 47% better parameter efficiency. Exploration also enables scaling generalization, and even scaling how end-to-end existing models are. As end-to-end generative models, XMs match diffusion on control tasks with up to 256× less inference compute.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2607.27372 in a model README.md to link it from this page.
Cite arxiv.org/abs/2607.27372 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2607.27372 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.