A comprehensive investigation of how to squeeze capacity from multimodal large language models for subject-driven generation, which requires both text instruction understanding and identity preservation for the subjects.</p>\n","updatedAt":"2026-05-27T02:17:07.089Z","author":{"_id":"64361417a4bd75c62cc1e534","avatarUrl":"/avatars/edf6ac3fb9bd63ceb78f3b95254e6b19.svg","fullname":"Shuhong Zheng","name":"ShuhongZheng","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9148783087730408},"editors":["ShuhongZheng"],"editorAvatarUrls":["/avatars/edf6ac3fb9bd63ceb78f3b95254e6b19.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2605.26111","authors":[{"_id":"6a1653bae9aa3c8e322db34f","user":{"_id":"64361417a4bd75c62cc1e534","avatarUrl":"/avatars/edf6ac3fb9bd63ceb78f3b95254e6b19.svg","isPro":false,"fullname":"Shuhong Zheng","user":"ShuhongZheng","type":"user","name":"ShuhongZheng"},"name":"Shuhong Zheng","status":"claimed_verified","statusLastChangedAt":"2026-05-27T07:41:23.500Z","hidden":false},{"_id":"6a1653bae9aa3c8e322db350","name":"Aashish Kumar Misraa","hidden":false},{"_id":"6a1653bae9aa3c8e322db351","name":"Yu-Teng Li","hidden":false},{"_id":"6a1653bae9aa3c8e322db352","name":"Yu-Jhe Li","hidden":false},{"_id":"6a1653bae9aa3c8e322db353","name":"Igor Gilitschenski","hidden":false}],"publishedAt":"2026-05-25T00:00:00.000Z","submittedOnDailyAt":"2026-05-27T00:00:00.000Z","title":"Squeezing Capacity from Multimodal Large Language Models for Subject-driven Generation","submittedOnDailyBy":{"_id":"64361417a4bd75c62cc1e534","avatarUrl":"/avatars/edf6ac3fb9bd63ceb78f3b95254e6b19.svg","isPro":false,"fullname":"Shuhong Zheng","user":"ShuhongZheng","type":"user","name":"ShuhongZheng"},"summary":"Subject-driven image generation aims to synthesize new images that preserve the identity of the given subject while following textual instructions. Existing approaches often encode text and reference images separately. This limits cross-modal reasoning abilities and causes copy-paste artifacts. Recent frameworks that connect multimodal models and diffusion models improve instruction following, but largely overlook identity preservation. To address these limitations, we condition diffusion models on Multimodal Large Language Models (MLLMs) that jointly encode text and reference images, and augment it with VAE-based identity conditioning. A novel Dual Layer Aggregation (DLA) module is designed to aggregate multi-level MLLM features for optimal conditioning, and a multi-stage denoising strategy is applied to progressively balance the semantic information from MLLM and fine-detail identity from VAE during inference. Extensive experiments demonstrate that our approach harmonizes multimodal understanding with identity preservation, mitigates copy-paste issues, and achieves superior performance regarding human preference on subject-driven image generation. Our project website is available at https://zsh2000.github.io/squeeze-mllm-subject-gen/.","upvotes":3,"discussionId":"6a1653bae9aa3c8e322db354","projectPage":"https://zsh2000.github.io/squeeze-mllm-subject-gen/","ai_summary":"A novel approach conditions diffusion models on multimodal large language models for subject-driven image generation, combining text and reference image encoding with VAE-based identity conditioning to improve both semantic understanding and identity preservation.","ai_keywords":["diffusion models","multimodal large language models","VAE-based identity conditioning","dual layer aggregation","multi-stage denoising strategy","cross-modal reasoning","copy-paste artifacts","semantic information","fine-detail identity"],"organization":{"_id":"61e5d14f77496de0a6d95c6b","name":"adobe","fullname":"Adobe","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1645217431826-61e35e517ac6b6d06cfa8081.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"64361417a4bd75c62cc1e534","avatarUrl":"/avatars/edf6ac3fb9bd63ceb78f3b95254e6b19.svg","isPro":false,"fullname":"Shuhong Zheng","user":"ShuhongZheng","type":"user"},{"_id":"6270324ebecab9e2dcf245de","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6270324ebecab9e2dcf245de/cMbtWSasyNlYc9hvsEEzt.jpeg","isPro":false,"fullname":"Kye Gomez","user":"kye","type":"user"},{"_id":"68e41658ed521a20f060f103","avatarUrl":"/avatars/e35cb262601a5ee932fcd3cbbe10be5e.svg","isPro":false,"fullname":"Shuhong Zheng","user":"zsh2000","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"61e5d14f77496de0a6d95c6b","name":"adobe","fullname":"Adobe","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1645217431826-61e35e517ac6b6d06cfa8081.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2605/2605.26111.md"}">

Get this paper in your agent:

Don't have the latest CLI?

Cite arxiv.org/abs/2605.26111 in a model README.md to link it from this page.

Cite arxiv.org/abs/2605.26111 in a dataset README.md to link it from this page.

Cite arxiv.org/abs/2605.26111 in a Space README.md to link it from this page.

Add this paper to a collection to link it from this page.