synthetic data generation with LLM-based API simulation</p>\n","updatedAt":"2026-07-21T02:47:10.620Z","author":{"_id":"64ad5f59b7e4b2c1ce47eb43","avatarUrl":"/avatars/1f13ebe21a90d8c99920aa2c8cd9ac45.svg","fullname":"Seanie Lee","name":"Seanie-lee","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":7,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8115416169166565},"editors":["Seanie-lee"],"editorAvatarUrls":["/avatars/1f13ebe21a90d8c99920aa2c8cd9ac45.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2607.16900","authors":[{"_id":"6a5edd934fe5d1d13e84ab51","name":"Seanie Lee","hidden":false},{"_id":"6a5edd934fe5d1d13e84ab52","name":"Sanjoy Chowdhury","hidden":false},{"_id":"6a5edd934fe5d1d13e84ab53","name":"Chao Jiang","hidden":false},{"_id":"6a5edd934fe5d1d13e84ab54","name":"Cheng-Yu Hsieh","hidden":false},{"_id":"6a5edd934fe5d1d13e84ab55","name":"Ting-Yao Hu","hidden":false},{"_id":"6a5edd934fe5d1d13e84ab56","name":"Alexander T Toshev","hidden":false},{"_id":"6a5edd934fe5d1d13e84ab57","name":"Oncel Tuzel","hidden":false},{"_id":"6a5edd934fe5d1d13e84ab58","name":"Raviteja Vemulapalli","hidden":false}],"publishedAt":"2026-07-18T00:00:00.000Z","submittedOnDailyAt":"2026-07-21T00:00:00.000Z","title":"Environment-free Synthetic Data Generation for API-Calling Agents","submittedOnDailyBy":{"_id":"64ad5f59b7e4b2c1ce47eb43","avatarUrl":"/avatars/1f13ebe21a90d8c99920aa2c8cd9ac45.svg","isPro":false,"fullname":"Seanie Lee","user":"Seanie-lee","type":"user","name":"Seanie-lee"},"summary":"Training API-calling large language model (LLM) agents demands massive amounts of high-quality trajectories. However, collecting such data at scale typically requires fully implemented environments with executable APIs and realistic, pre-populated backend databases, creating a major bottleneck for scalability. To overcome this, we propose an environment-free synthetic data generation approach that leverages LLMs as on-the-fly digital world models. Given only API specifications, our method generates trajectories mimicking interactions between an agent and a stateful environment. Specifically, an LLM first generates diverse tasks solvable with the provided APIs. A teacher agent then iteratively solves each task while an LLM simulator generates coherent synthetic API responses conditioned on the task context and simulation history. Finally, an LLM judge filters the trajectories to ensure the quality of the resulting dataset. We evaluate our approach on the challenging AppWorld and OfficeBench benchmarks, which include both information-retrieval and state-changing tasks. Fine-tuning models on our synthetic data yields significant performance gains, demonstrating that effective supervision for API-calling agents can be generated without any executable environment. Our results establish LLM-based API simulation as a practical, scalable solution for training agents across diverse API ecosystems.","upvotes":9,"discussionId":"6a5edd934fe5d1d13e84ab59","organization":{"_id":"628cbd99ef14f971b69948ab","name":"apple","fullname":"Apple","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1653390727490-5dd96eb166059660ed1ee413.jpeg"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6039478ab3ecf716b1a5fd4d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6039478ab3ecf716b1a5fd4d/_Thy4E7taiSYBLKxEKJbT.jpeg","isPro":true,"fullname":"taesiri","user":"taesiri","type":"user"},{"_id":"60f1abe7544c2adfd699860c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674929746905-60f1abe7544c2adfd699860c.jpeg","isPro":false,"fullname":"AK","user":"akhaliq","type":"user"},{"_id":"65ff6b4f3d247974baba6972","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65ff6b4f3d247974baba6972/-EPxFSSIWNikdakI750az.jpeg","isPro":false,"fullname":"Vinh Nguyen","user":"vinhnx90","type":"user"},{"_id":"6a2da6c8ca070ee12c6e396c","avatarUrl":"/avatars/0355287dcabaa67dbc7f0b10b87451f9.svg","isPro":false,"fullname":"Joe Mama","user":"JoeMama123123123","type":"user"},{"_id":"638716c14e00d7fc0902fef4","avatarUrl":"/avatars/5fa8152f8c0e4e600d1a64802c3e0103.svg","isPro":false,"fullname":"Sangwoo Park","user":"Sangsang","type":"user"},{"_id":"64b74920fe6a108d03fed767","avatarUrl":"/avatars/a2c05b809c36fa5fab8e1a43b3e67051.svg","isPro":false,"fullname":"Minki Kang","user":"Nardien","type":"user"},{"_id":"66d30f5fad293ffc4b7672bc","avatarUrl":"/avatars/6f164d813b947940a088820f8fd4dbe8.svg","isPro":false,"fullname":"Woongyeong Yeo","user":"wgcyeo","type":"user"},{"_id":"64ad5f59b7e4b2c1ce47eb43","avatarUrl":"/avatars/1f13ebe21a90d8c99920aa2c8cd9ac45.svg","isPro":false,"fullname":"Seanie Lee","user":"Seanie-lee","type":"user"},{"_id":"63ac5701c21e60a3e9b58aa7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63ac5701c21e60a3e9b58aa7/g6EX7diOpuA94R2ab-rZC.png","isPro":true,"fullname":"Dipankar Sarkar","user":"dipankarsarkar","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"628cbd99ef14f971b69948ab","name":"apple","fullname":"Apple","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/1653390727490-5dd96eb166059660ed1ee413.jpeg"},"query":{}}">
Environment-free Synthetic Data Generation for API-Calling Agents
Abstract
Training API-calling large language model (LLM) agents demands massive amounts of high-quality trajectories. However, collecting such data at scale typically requires fully implemented environments with executable APIs and realistic, pre-populated backend databases, creating a major bottleneck for scalability. To overcome this, we propose an environment-free synthetic data generation approach that leverages LLMs as on-the-fly digital world models. Given only API specifications, our method generates trajectories mimicking interactions between an agent and a stateful environment. Specifically, an LLM first generates diverse tasks solvable with the provided APIs. A teacher agent then iteratively solves each task while an LLM simulator generates coherent synthetic API responses conditioned on the task context and simulation history. Finally, an LLM judge filters the trajectories to ensure the quality of the resulting dataset. We evaluate our approach on the challenging AppWorld and OfficeBench benchmarks, which include both information-retrieval and state-changing tasks. Fine-tuning models on our synthetic data yields significant performance gains, demonstrating that effective supervision for API-calling agents can be generated without any executable environment. Our results establish LLM-based API simulation as a practical, scalable solution for training agents across diverse API ecosystems.
Community
synthetic data generation with LLM-based API simulation
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2607.16900 in a model README.md to link it from this page.
Cite arxiv.org/abs/2607.16900 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2607.16900 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.