A diagnosis-driven framework for iterative prompt tuning with function-calling LLMs.</p>\n","updatedAt":"2026-05-29T23:31:13.320Z","author":{"_id":"64bcf5d286e7fb5b8a59f314","avatarUrl":"/avatars/2a065aaf1941a59a98508ed0960c5b8e.svg","fullname":"Farima Fatahi","name":"farimafatahi","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7117559313774109},"editors":["farimafatahi"],"editorAvatarUrls":["/avatars/2a065aaf1941a59a98508ed0960c5b8e.svg"],"reactions":[],"isReport":false}},{"id":"6a1a40afa311d2bf1543c2f2","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":359,"isUserFollowing":false},"createdAt":"2026-05-30T01:43:11.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [JTPRO: A Joint Tool-Prompt Reflective Optimization Framework for Language Agents](https://huggingface.co/papers/2604.19821) (2026)\n* [Adaptive Prompt Structure Factorization: A Framework for Self-Discovering and Optimizing Compositional Prompt Programs](https://huggingface.co/papers/2604.06699) (2026)\n* [CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimization](https://huggingface.co/papers/2604.14214) (2026)\n* [When Correct Isn't Usable: Improving Structured Output Reliability in Small Language Models](https://huggingface.co/papers/2605.02363) (2026)\n* [Prompting Policies for Multi-step Reasoning and Tool-Use in Black-box LLMs with Iterative Distillation of Experience](https://huggingface.co/papers/2605.14443) (2026)\n* [Prompt Codebooks: Discrete Compositional Optimization for Language Model Instruction Refinement](https://huggingface.co/papers/2605.28360) (2026)\n* [Agent-GWO: Collaborative Agents for Dynamic Prompt Optimization in Large Language Models](https://huggingface.co/papers/2604.18612) (2026)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"<p>This is an automated message from the <a href=\"https://huggingface.co/librarian-bots\">Librarian Bot</a>. I found the following papers similar to this paper. </p>\n<p>The following papers were recommended by the Semantic Scholar API </p>\n<ul>\n<li><a href=\"https://huggingface.co/papers/2604.19821\">JTPRO: A Joint Tool-Prompt Reflective Optimization Framework for Language Agents</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2604.06699\">Adaptive Prompt Structure Factorization: A Framework for Self-Discovering and Optimizing Compositional Prompt Programs</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2604.14214\">CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimization</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.02363\">When Correct Isn't Usable: Improving Structured Output Reliability in Small Language Models</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.14443\">Prompting Policies for Multi-step Reasoning and Tool-Use in Black-box LLMs with Iterative Distillation of Experience</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2605.28360\">Prompt Codebooks: Discrete Compositional Optimization for Language Model Instruction Refinement</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2604.18612\">Agent-GWO: Collaborative Agents for Dynamic Prompt Optimization in Large Language Models</a> (2026)</li>\n</ul>\n<p> Please give a thumbs up to this comment if you found it helpful!</p>\n<p> If you want recommendations for any Paper on Hugging Face checkout <a href=\"https://huggingface.co/spaces/librarian-bots/recommend_similar_papers\">this</a> Space</p>\n<p> You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: <code><span class=\"SVELTE_PARTIAL_HYDRATER contents\" data-target=\"UserMention\" data-props=\"{&quot;user&quot;:&quot;librarian-bot&quot;}\"><span class=\"inline-block\"><span class=\"contents\"><a href=\"/librarian-bot\">@<span class=\"underline\">librarian-bot</span></a></span> </span></span> recommend</code></p>\n","updatedAt":"2026-05-30T01:43:11.523Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":359,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.7097659111022949},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2605.21781","authors":[{"_id":"6a1a20ce808ddbc3c7d42ec2","name":"Farima Fatahi Bayat","hidden":false},{"_id":"6a1a20ce808ddbc3c7d42ec3","name":"Moin Aminnaseri","hidden":false},{"_id":"6a1a20ce808ddbc3c7d42ec4","name":"Pouya Pezeshkpour","hidden":false},{"_id":"6a1a20ce808ddbc3c7d42ec5","name":"Estevam Hruschka","hidden":false}],"publishedAt":"2026-05-20T00:00:00.000Z","submittedOnDailyAt":"2026-05-29T00:00:00.000Z","title":"Reflective Prompt Tuning through Language Model Function-Calling","submittedOnDailyBy":{"_id":"64bcf5d286e7fb5b8a59f314","avatarUrl":"/avatars/2a065aaf1941a59a98508ed0960c5b8e.svg","isPro":false,"fullname":"Farima Fatahi","user":"farimafatahi","type":"user","name":"farimafatahi"},"summary":"Large language models (LLMs) have become increasingly capable of following instructions and complex reasoning, making prompting a flexible interface for adapting models without parameter updates. Yet prompt design remains labor-intensive and highly sensitive to formatting, phrasing, and instruction order, motivating automated prompt optimization methods that reduce manual effort while preserving inference-time flexibility. However, existing methods often search over prompt candidates or use fixed critique-refine pipelines driven by individual examples or small batches, limiting their ability to capture systematic error patterns and make targeted edits grounded in failure history. We propose Reflective Prompt Tuning (RPT), a framework that uses LLM function calling to simulate the iterative workflow of human prompt engineers. An LLM optimizer calls a diagnostic function that evaluates the target model over an entire optimization set, summarizes recurring failure modes, and returns a structured diagnostic report. The optimizer uses this report, together with an accumulated memory of prior reports, to revise the prompt for the next iteration. RPT further supports confidence-aware optimization by using calibration signals in diagnostic feedback and final prompt selection. Across three reasoning tasks, RPT improves over initial prompts by up to 12.9 points, remains competitive with state of the art, and improves confidence calibration. Our analyses show that RPT is especially effective on multi-hop and mathematical reasoning, producing targeted prompt revisions that align with diagnosed failure patterns and lead to gains in task performance and calibration.","upvotes":2,"discussionId":"6a1a20cf808ddbc3c7d42ec6","githubRepo":"https://github.com/megagonlabs/RPT","githubRepoAddedBy":"user","ai_summary":"Reflective Prompt Tuning (RPT) automates prompt optimization for large language models by simulating human iterative engineering through diagnostic feedback and memory-based revision cycles.","ai_keywords":["large language models","prompt tuning","function calling","diagnostic function","iterative workflow","optimization set","failure modes","calibration signals","confidence-aware optimization","multi-hop reasoning","mathematical reasoning"],"githubStars":0,"organization":{"_id":"611b3430ffd776a3dbc6a1ca","name":"megagon","fullname":"Megagon Labs","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1631300650923-5fc181c4ea82dd667bb0ffae.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"64bcf5d286e7fb5b8a59f314","avatarUrl":"/avatars/2a065aaf1941a59a98508ed0960c5b8e.svg","isPro":false,"fullname":"Farima Fatahi","user":"farimafatahi","type":"user"},{"_id":"6270324ebecab9e2dcf245de","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6270324ebecab9e2dcf245de/cMbtWSasyNlYc9hvsEEzt.jpeg","isPro":false,"fullname":"Kye Gomez","user":"kye","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"611b3430ffd776a3dbc6a1ca","name":"megagon","fullname":"Megagon Labs","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1631300650923-5fc181c4ea82dd667bb0ffae.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2605/2605.21781.md"}">

Get this paper in your agent:

Don't have the latest CLI?

Cite arxiv.org/abs/2605.21781 in a model README.md to link it from this page.

Cite arxiv.org/abs/2605.21781 in a dataset README.md to link it from this page.

Cite arxiv.org/abs/2605.21781 in a Space README.md to link it from this page.

Add this paper to a collection to link it from this page.