Hugging Face Daily Papers · · 3 min read

Teaching LLMs a Low-Resource Language: Enhancing Code Completion in Pharo

Mirrored from Hugging Face Daily Papers for archival readability. Support the source by reading on the original site.

.</p>\n","updatedAt":"2026-07-09T06:04:57.764Z","author":{"_id":"663486a1f64712540644cb68","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/663486a1f64712540644cb68/YZFR41ERY6UrC6rCC6Nan.jpeg","fullname":"Alessandro","name":"Devy1","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":4,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"fr","probability":0.32275810837745667},"editors":["Devy1"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/663486a1f64712540644cb68/YZFR41ERY6UrC6rCC6Nan.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2607.04939","authors":[{"_id":"6a4c9e1f25849b193a834386","name":"Kilian Kier","hidden":false},{"_id":"6a4c9e1f25849b193a834387","user":{"_id":"663486a1f64712540644cb68","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/663486a1f64712540644cb68/YZFR41ERY6UrC6rCC6Nan.jpeg","isPro":false,"fullname":"Alessandro","user":"Devy1","type":"user","name":"Devy1"},"name":"Alessandro Giagnorio","status":"claimed_verified","statusLastChangedAt":"2026-07-07T08:30:20.729Z","hidden":false},{"_id":"6a4c9e1f25849b193a834388","name":"Omar AbedelKader","hidden":false},{"_id":"6a4c9e1f25849b193a834389","name":"Oleksandr Zaitsev","hidden":false},{"_id":"6a4c9e1f25849b193a83438a","name":"Robert Peharz","hidden":false},{"_id":"6a4c9e1f25849b193a83438b","name":"Romain Robbes","hidden":false},{"_id":"6a4c9e1f25849b193a83438c","name":"Gabriele Bavota","hidden":false},{"_id":"6a4c9e1f25849b193a83438d","name":"Stéphane Ducasse","hidden":false}],"publishedAt":"2026-07-06T00:00:00.000Z","submittedOnDailyAt":"2026-07-09T00:00:00.000Z","title":"Teaching LLMs a Low-Resource Language: Enhancing Code Completion in Pharo","submittedOnDailyBy":{"_id":"663486a1f64712540644cb68","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/663486a1f64712540644cb68/YZFR41ERY6UrC6rCC6Nan.jpeg","isPro":false,"fullname":"Alessandro","user":"Devy1","type":"user","name":"Devy1"},"summary":"Large Language Models (LLMs) unlocked new possibilities in automated code writing, becoming the backbone of most code completion tools. While LLMs excel in mainstream languages, they often lack support for the so-called low-resource languages where training data is scarce. As a result, these languages lag behind in the quality of code completion tooling available to their communities. A concrete example is Pharo, a Smalltalk-inspired language whose IDE currently offers only single-token completion. In this work, we report on our experience bringing LLM-based code completion to Pharo. First, we describe an end-to-end pipeline that combines Pharo-specific data curation, continued pre-training and fine-tuning of open code LLMs. Second, we introduce a set of Pharo code completion benchmarks designed to evaluate whether models (i) learn Pharo's syntax and (ii) accurately complete masked Pharo code from real-world GitHub repositories. Third, we show empirically that Pharo-specialized models substantially outperform their original base checkpoints and also exceed the accuracy of substantially larger code LLMs on Pharo completion. Overall, our case study demonstrates the feasibility of bringing strong LLM-based code completion to low-resource programming languages, with models small enough to provide ``real-time'' in-IDE support.","upvotes":1,"discussionId":"6a4c9e1f25849b193a83438e","githubRepo":"https://github.com/kilian-kier/pharo-llm-completion","githubRepoAddedBy":"user","ai_summary":"Large language models can be adapted for low-resource programming languages through specialized training pipelines and benchmarks, achieving superior code completion performance compared to general-purpose models.","ai_keywords":["large language models","code completion","continued pre-training","fine-tuning","code completion benchmarks","Pharo language","real-time in-IDE support"],"ai_summary_model":"Qwen/Qwen2.5-Coder-32B-Instruct","githubStars":0},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"697c8b15a7f796854ef333c4","avatarUrl":"/avatars/94de3a736fac914944f1b57609e3819a.svg","isPro":false,"fullname":"Joel Wang","user":"joelhenwang","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"query":{}}">
Papers
arxiv:2607.04939

Teaching LLMs a Low-Resource Language: Enhancing Code Completion in Pharo

Published on Jul 6
· Submitted by
Alessandro
on Jul 9
Authors:
,

Abstract

Large language models can be adapted for low-resource programming languages through specialized training pipelines and benchmarks, achieving superior code completion performance compared to general-purpose models.

Large Language Models (LLMs) unlocked new possibilities in automated code writing, becoming the backbone of most code completion tools. While LLMs excel in mainstream languages, they often lack support for the so-called low-resource languages where training data is scarce. As a result, these languages lag behind in the quality of code completion tooling available to their communities. A concrete example is Pharo, a Smalltalk-inspired language whose IDE currently offers only single-token completion. In this work, we report on our experience bringing LLM-based code completion to Pharo. First, we describe an end-to-end pipeline that combines Pharo-specific data curation, continued pre-training and fine-tuning of open code LLMs. Second, we introduce a set of Pharo code completion benchmarks designed to evaluate whether models (i) learn Pharo's syntax and (ii) accurately complete masked Pharo code from real-world GitHub repositories. Third, we show empirically that Pharo-specialized models substantially outperform their original base checkpoints and also exceed the accuracy of substantially larger code LLMs on Pharo completion. Overall, our case study demonstrates the feasibility of bringing strong LLM-based code completion to low-resource programming languages, with models small enough to provide ``real-time'' in-IDE support.

Community

Paper author Paper submitter about 4 hours ago
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images

· Sign up or log in to comment

Models citing this paper

No model linking this paper

Cite arxiv.org/abs/2607.04939 in a model README.md to link it from this page.

Datasets citing this paper

No dataset linking this paper

Cite arxiv.org/abs/2607.04939 in a dataset README.md to link it from this page.

Spaces citing this paper

No Space linking this paper

Cite arxiv.org/abs/2607.04939 in a Space README.md to link it from this page.

Collections including this paper

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from Hugging Face Daily Papers