Vision-Language-Action (VLA) models widely adopt pretrained Vision-Language Models (VLMs) as policy backbones, yet it remains unclear what kind of pretrained VLM representation is useful as a VLA initialization. In this paper, we study VLA initialization as a controlled representation-design problem along three axes: capability-level embodied VQA supervision, parameter-update strategy, and robot-data pretraining. Our experiments show that the original pretrained VLM representation is a key source of action performance. However, embodied VQA adaptation does not yield uniform gains: its benefit depends on downstream bottlenecks, and gains from different capability domains are not simply additive.<br>For update strategy, LoRA provides a more reliable initialization than Full Finetune, indicating that overly reshaping the pretrained representation can weaken VLA initialization. Robot-data pretraining further improves VLA initialization, with the strongest variant obtained by staged LoRA-based training. Together, these findings suggest that effective VLM-to-VLA adaptation should inject action-relevant embodied and robot-trajectory signals while preserving the pretrained VLM representation that remains useful for action learning.</p>\n","updatedAt":"2026-05-27T07:59:55.408Z","author":{"_id":"66026c9068d519ed32519e9c","avatarUrl":"/avatars/8fa051312c713772e5b8ba65989ff7f5.svg","fullname":"Weifeng Lin","name":"Afeng-x","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":5,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9117465019226074},"editors":["Afeng-x"],"editorAvatarUrls":["/avatars/8fa051312c713772e5b8ba65989ff7f5.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2605.25802","authors":[{"_id":"6a15559ab57a1823d5708de0","name":"Weifeng Lin","hidden":false},{"_id":"6a15559ab57a1823d5708de1","name":"Siyuan Huang","hidden":false},{"_id":"6a15559ab57a1823d5708de2","name":"Hao Li","hidden":false},{"_id":"6a15559ab57a1823d5708de3","name":"Tingwei Chen","hidden":false},{"_id":"6a15559ab57a1823d5708de4","name":"Ruichuan An","hidden":false},{"_id":"6a15559ab57a1823d5708de5","name":"Xinyu Wei","hidden":false},{"_id":"6a15559ab57a1823d5708de6","name":"Jianbo Liu","hidden":false},{"_id":"6a15559ab57a1823d5708de7","name":"Hongsheng Li","hidden":false}],"publishedAt":"2026-05-25T00:00:00.000Z","submittedOnDailyAt":"2026-05-27T00:00:00.000Z","title":"Rethinking VLM Representation for VLA Initialization","submittedOnDailyBy":{"_id":"66026c9068d519ed32519e9c","avatarUrl":"/avatars/8fa051312c713772e5b8ba65989ff7f5.svg","isPro":false,"fullname":"Weifeng Lin","user":"Afeng-x","type":"user","name":"Afeng-x"},"summary":"Vision-Language-Action (VLA) models widely adopt pretrained Vision-Language Models (VLMs) as policy backbones, yet it remains unclear what kind of pretrained VLM representation is useful as a VLA initialization. In this paper, we study VLA initialization as a controlled representation-design problem along three axes: capability-level embodied VQA supervision, parameter-update strategy, and robot-data pretraining. Our experiments show that the original pretrained VLM representation is a key source of action performance. However, embodied VQA adaptation does not yield uniform gains: its benefit depends on downstream bottlenecks, and gains from different capability domains are not simply additive. For update strategy, LoRA provides a more reliable initialization than Full Finetune, indicating that overly reshaping the pretrained representation can weaken VLA initialization. Robot-data pretraining further improves VLA initialization, with the strongest variant obtained by staged LoRA-based training. Together, these findings suggest that effective VLM-to-VLA adaptation should inject action-relevant embodied and robot-trajectory signals while preserving the pretrained VLM representation that remains useful for action learning.","upvotes":6,"discussionId":"6a15559ab57a1823d5708de8","projectPage":"https://github.com/AFeng-x/Rethink_VLA_Initialization","githubRepo":"https://github.com/AFeng-x/Rethink_VLA_Initialization","githubRepoAddedBy":"user","ai_summary":"Effective vision-language-action model initialization requires balancing pretrained vision-language model representations with embodied task-specific adaptations and robot-data pretraining while preserving core action-relevant features.","ai_keywords":["Vision-Language-Action models","Vision-Language Models","embodied VQA","parameter-update strategy","LoRA","Full Finetune","robot-data pretraining","staged training"],"githubStars":2,"organization":{"_id":"6390c6fdd00f25601f445cd4","name":"CUHK-CSE","fullname":"The Chinese University of Hong Kong","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/621f2eb36e152b56a7cf0248/o8RRAczRjfNEzq70GzUwQ.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"66026c9068d519ed32519e9c","avatarUrl":"/avatars/8fa051312c713772e5b8ba65989ff7f5.svg","isPro":false,"fullname":"Weifeng Lin","user":"Afeng-x","type":"user"},{"_id":"66f29285ea3b2dd35675c3b8","avatarUrl":"/avatars/c7c28fe55e16b00700e7ebf9a7303ac3.svg","isPro":false,"fullname":"Allen Wei","user":"AllenWei1116","type":"user"},{"_id":"6537e7e55ad6715c4c43297b","avatarUrl":"/avatars/069e4afb7efdbef0c467461e8d390bc9.svg","isPro":false,"fullname":"zhengyuanhong","user":"zyh200727","type":"user"},{"_id":"6895e7f146763431aea25ca4","avatarUrl":"/avatars/52e550c3f7e8da2e31b63413e2e71e6c.svg","isPro":false,"fullname":"Tianyi Jiang","user":"LumosJiang","type":"user"},{"_id":"651a4571ea991a32291283c6","avatarUrl":"/avatars/b4e358e68bcc56d470a34a14d16c56fb.svg","isPro":false,"fullname":"HankYang","user":"HankYang428","type":"user"},{"_id":"634e4120038b5879133552f5","avatarUrl":"/avatars/34ec861b4bbf1aecf927a7d6e726c7a4.svg","isPro":true,"fullname":"Siyuan","user":"SiyuanH","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"6390c6fdd00f25601f445cd4","name":"CUHK-CSE","fullname":"The Chinese University of Hong Kong","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/621f2eb36e152b56a7cf0248/o8RRAczRjfNEzq70GzUwQ.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2605/2605.25802.md"}">
Rethinking VLM Representation for VLA Initialization
Abstract
Effective vision-language-action model initialization requires balancing pretrained vision-language model representations with embodied task-specific adaptations and robot-data pretraining while preserving core action-relevant features.
AI-generated summary
Vision-Language-Action (VLA) models widely adopt pretrained Vision-Language Models (VLMs) as policy backbones, yet it remains unclear what kind of pretrained VLM representation is useful as a VLA initialization. In this paper, we study VLA initialization as a controlled representation-design problem along three axes: capability-level embodied VQA supervision, parameter-update strategy, and robot-data pretraining. Our experiments show that the original pretrained VLM representation is a key source of action performance. However, embodied VQA adaptation does not yield uniform gains: its benefit depends on downstream bottlenecks, and gains from different capability domains are not simply additive. For update strategy, LoRA provides a more reliable initialization than Full Finetune, indicating that overly reshaping the pretrained representation can weaken VLA initialization. Robot-data pretraining further improves VLA initialization, with the strongest variant obtained by staged LoRA-based training. Together, these findings suggest that effective VLM-to-VLA adaptation should inject action-relevant embodied and robot-trajectory signals while preserving the pretrained VLM representation that remains useful for action learning.
Community
Vision-Language-Action (VLA) models widely adopt pretrained Vision-Language Models (VLMs) as policy backbones, yet it remains unclear what kind of pretrained VLM representation is useful as a VLA initialization. In this paper, we study VLA initialization as a controlled representation-design problem along three axes: capability-level embodied VQA supervision, parameter-update strategy, and robot-data pretraining. Our experiments show that the original pretrained VLM representation is a key source of action performance. However, embodied VQA adaptation does not yield uniform gains: its benefit depends on downstream bottlenecks, and gains from different capability domains are not simply additive.
For update strategy, LoRA provides a more reliable initialization than Full Finetune, indicating that overly reshaping the pretrained representation can weaken VLA initialization. Robot-data pretraining further improves VLA initialization, with the strongest variant obtained by staged LoRA-based training. Together, these findings suggest that effective VLM-to-VLA adaptation should inject action-relevant embodied and robot-trajectory signals while preserving the pretrained VLM representation that remains useful for action learning.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2605.25802 in a model README.md to link it from this page.
Cite arxiv.org/abs/2605.25802 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2605.25802 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.