.</p>\n","updatedAt":"2026-06-19T09:03:34.024Z","author":{"_id":"663486a1f64712540644cb68","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/663486a1f64712540644cb68/YZFR41ERY6UrC6rCC6Nan.jpeg","fullname":"Alessandro","name":"Devy1","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":4,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"fr","probability":0.32275810837745667},"editors":["Devy1"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/663486a1f64712540644cb68/YZFR41ERY6UrC6rCC6Nan.jpeg"],"reactions":[],"isReport":false}},{"id":"6a3584728567badf759a0c86","author":{"_id":"6960eca92f7ad9b043b5cbe0","avatarUrl":"/avatars/e68dcc7fd04f143d849d40414866e633.svg","fullname":"Noah","name":"noahml","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":0,"isUserFollowing":false},"createdAt":"2026-06-19T18:03:30.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"Cool paper - I liked the way \"No Resource, No Benchmarks, No Problem? Evaluating and Improving LLMs for Code Generation in No-Resource Languages\" frames the problem without making it feel too abstract.\n\nCurious if you think this would still work once the setup gets messier in the wild?\n\nI made a podcast on it with ResearchPod, it makes it easy to get the key concepts on the go:\nhttps://researchpod.app/episode/135c71ee-125b-43bd-b92c-bebd114dfdbe","html":"<p>Cool paper - I liked the way \"No Resource, No Benchmarks, No Problem? Evaluating and Improving LLMs for Code Generation in No-Resource Languages\" frames the problem without making it feel too abstract.</p>\n<p>Curious if you think this would still work once the setup gets messier in the wild?</p>\n<p>I made a podcast on it with ResearchPod, it makes it easy to get the key concepts on the go:<br><a href=\"https://researchpod.app/episode/135c71ee-125b-43bd-b92c-bebd114dfdbe\" rel=\"nofollow\">https://researchpod.app/episode/135c71ee-125b-43bd-b92c-bebd114dfdbe</a></p>\n","updatedAt":"2026-06-19T18:03:30.120Z","author":{"_id":"6960eca92f7ad9b043b5cbe0","avatarUrl":"/avatars/e68dcc7fd04f143d849d40414866e633.svg","fullname":"Noah","name":"noahml","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":0,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8500277400016785},"editors":["noahml"],"editorAvatarUrls":["/avatars/e68dcc7fd04f143d849d40414866e633.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2606.16827","authors":[{"_id":"6a33930559127a45e2c1c6dc","user":{"_id":"663486a1f64712540644cb68","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/663486a1f64712540644cb68/YZFR41ERY6UrC6rCC6Nan.jpeg","isPro":false,"fullname":"Alessandro","user":"Devy1","type":"user","name":"Devy1"},"name":"Alessandro Giagnorio","status":"claimed_verified","statusLastChangedAt":"2026-06-18T11:26:26.695Z","hidden":false},{"_id":"6a33930559127a45e2c1c6dd","name":"Alberto Martin-Lopez","hidden":false},{"_id":"6a33930559127a45e2c1c6de","name":"Gabriele Bavota","hidden":false}],"publishedAt":"2026-06-15T00:00:00.000Z","submittedOnDailyAt":"2026-06-19T00:00:00.000Z","title":"No Resource, No Benchmarks, No Problem? Evaluating and Improving LLMs for Code Generation in No-Resource Languages","submittedOnDailyBy":{"_id":"663486a1f64712540644cb68","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/663486a1f64712540644cb68/YZFR41ERY6UrC6rCC6Nan.jpeg","isPro":false,"fullname":"Alessandro","user":"Devy1","type":"user","name":"Devy1"},"summary":"Large Language Models (LLMs) have significantly advanced the automation of software engineering tasks. One prominent example is code generation, where an LLM produces code in a specified programming language based on a natural language description. Most research in this area has focused on high-resource languages, such as Python or Java, which benefit from abundant training data. A smaller body of work has explored low-resource languages, which are underrepresented in training corpora. In contrast, no-resource languages for which LLMs have seen virtually no training data remain largely unstudied. These languages often emerge in industry, where organizations develop proprietary or domain-specific languages unsupported by commercial tools like GitHub Copilot. This results in the need for companies to deploy their own in-house code recommenders. To investigate possible solutions in this context, we build and release three code generation benchmarks for no-resource languages, based on two recently proposed programming languages for which very little training data is available. Using these benchmarks, we experiment several solutions to teach LLMs about no-resource languages, including prompt-based techniques as well as pre-training and fine-tuning exploiting the little data available. While further pre-training gives the largest performance gains for no-resource languages, applying it directly to instruction-tuned models harms their ability to follow instructions. To address this, we start from a base model, further pre-training it on the target language, and then inject instruction-following capabilities via weight diff transfer from an instruction model. Such an approach significantly improves code generation capabilities in no-resource settings, allowing companies to cheaply deploy a specialized instruct model without dealing with the computational cost of instruction fine-tuning.","upvotes":0,"discussionId":"6a33930559127a45e2c1c6df","githubRepo":"https://github.com/Devy99/no-resource-pl-study","githubRepoAddedBy":"user","ai_summary":"Research addresses code generation challenges for no-resource programming languages by developing benchmarks and proposing a method that combines further pre-training with weight difference transfer to create specialized instruction-following models at reduced computational cost.","ai_keywords":["large language models","code generation","no-resource languages","prompt-based techniques","pre-training","fine-tuning","instruction-tuned models","weight diff transfer","instruction-following capabilities"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","githubStars":1},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[],"acceptLanguages":["en"],"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2606/2606.16827.md","query":{}}">
No Resource, No Benchmarks, No Problem? Evaluating and Improving LLMs for Code Generation in No-Resource Languages
Abstract
Research addresses code generation challenges for no-resource programming languages by developing benchmarks and proposing a method that combines further pre-training with weight difference transfer to create specialized instruction-following models at reduced computational cost.
Large Language Models (LLMs) have significantly advanced the automation of software engineering tasks. One prominent example is code generation, where an LLM produces code in a specified programming language based on a natural language description. Most research in this area has focused on high-resource languages, such as Python or Java, which benefit from abundant training data. A smaller body of work has explored low-resource languages, which are underrepresented in training corpora. In contrast, no-resource languages for which LLMs have seen virtually no training data remain largely unstudied. These languages often emerge in industry, where organizations develop proprietary or domain-specific languages unsupported by commercial tools like GitHub Copilot. This results in the need for companies to deploy their own in-house code recommenders. To investigate possible solutions in this context, we build and release three code generation benchmarks for no-resource languages, based on two recently proposed programming languages for which very little training data is available. Using these benchmarks, we experiment several solutions to teach LLMs about no-resource languages, including prompt-based techniques as well as pre-training and fine-tuning exploiting the little data available. While further pre-training gives the largest performance gains for no-resource languages, applying it directly to instruction-tuned models harms their ability to follow instructions. To address this, we start from a base model, further pre-training it on the target language, and then inject instruction-following capabilities via weight diff transfer from an instruction model. Such an approach significantly improves code generation capabilities in no-resource settings, allowing companies to cheaply deploy a specialized instruct model without dealing with the computational cost of instruction fine-tuning.
Community
Cool paper - I liked the way "No Resource, No Benchmarks, No Problem? Evaluating and Improving LLMs for Code Generation in No-Resource Languages" frames the problem without making it feel too abstract.
Curious if you think this would still work once the setup gets messier in the wild?
I made a podcast on it with ResearchPod, it makes it easy to get the key concepts on the go:
https://researchpod.app/episode/135c71ee-125b-43bd-b92c-bebd114dfdbe
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2606.16827 in a model README.md to link it from this page.
Cite arxiv.org/abs/2606.16827 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2606.16827 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.