Accepted to EMNLP 2026 (Main Conference)</p>\n","updatedAt":"2026-09-10T10:45:52.830Z","author":{"_id":"672288dd7055eec76d377268","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/672288dd7055eec76d377268/bBQ51VwKZJIkHkNffuEuJ.jpeg","fullname":"Yuan Gao","name":"Yuan-avs","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":3,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.8243268132209778},"editors":["Yuan-avs"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/672288dd7055eec76d377268/bBQ51VwKZJIkHkNffuEuJ.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2609.08965","authors":[{"_id":"6aa28892a2aeb74440b1e060","user":{"_id":"672288dd7055eec76d377268","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/672288dd7055eec76d377268/bBQ51VwKZJIkHkNffuEuJ.jpeg","isPro":true,"fullname":"Yuan Gao","user":"Yuan-avs","type":"user","name":"Yuan-avs"},"name":"Yuan Gao","status":"claimed_verified","statusLastChangedAt":"2026-09-10T16:45:04.698Z","hidden":false},{"_id":"6aa28892a2aeb74440b1e061","name":"Sebastian Müller","hidden":false},{"_id":"6aa28892a2aeb74440b1e062","name":"Mattia Piccinini","hidden":false},{"_id":"6aa28892a2aeb74440b1e063","name":"Marc Kaufeld","hidden":false},{"_id":"6aa28892a2aeb74440b1e064","name":"Yuchen Zhang","hidden":false},{"_id":"6aa28892a2aeb74440b1e065","name":"Finn Rasmus Schäfer","hidden":false},{"_id":"6aa28892a2aeb74440b1e066","name":"Qunying Song","hidden":false},{"_id":"6aa28892a2aeb74440b1e067","name":"Johannes Betz","hidden":false}],"mediaUrls":["https://cdn-uploads.huggingface.co/production/uploads/672288dd7055eec76d377268/Lioug17CyUkFav3zCMMp7.png"],"publishedAt":"2026-09-08T00:00:00.000Z","submittedOnDailyAt":"2026-09-10T00:00:00.000Z","title":"PlannerForge: LLM Agents for Scenario-Based Testing of Motion Planners in Autonomous Driving","submittedOnDailyBy":{"_id":"672288dd7055eec76d377268","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/672288dd7055eec76d377268/bBQ51VwKZJIkHkNffuEuJ.jpeg","isPro":true,"fullname":"Yuan Gao","user":"Yuan-avs","type":"user","name":"Yuan-avs"},"summary":"Ensuring the safety of autonomous driving is a critical challenge. Scenario-based testing is a systematic process used to validate Autonomous Driving Systems (ADSs), but it remains a fragmented modular pipeline in which scenario generation, retrieval, modification, ADS execution, and results analysis are performed by separate tools with little interaction. Large Language Model (LLM) agents have shown promise across ADS sub-systems such as perception, planning, and control. However, no prior work covers the whole scenario-based testing pipeline for ADSs with a unified LLM-agent framework. We present PlannerForge, an LLM-agent framework that extends all scenario-based testing stages (from Scenario Generation to ADS Assessment) and adds two further LLM-enhanced stages: ADS Enhancement and ADS Benchmarking. We evaluate PlannerForge with 10 off-the-shelf LLMs across all tasks (Generation, Selection, Modification, Module Routing, Planner Testing, and Enhancement) under 5 prompt conditions. Best-per-task scores range from 0.88 to 1.00, and open-source 20-35B backends match commercial APIs on most tasks. Open-source models such as Qwen3.6:35B match commercial APIs on three of the five tasks. Chaining the modules end-to-end retains 83% / 78% of seed queries (commercial / open). It outperforms Scenario Factory 2.0 (Finkeldei et al., 2025) on natural-language generation (193 vs. 144 executable of 200) and realises 92-96% of requested city, road and vehicle attributes. It outperforms BM25 (Robertson and Zaragoza, 2009) at rank 1 selection (92.0% vs. 67.5%) and From-Words-to-Collisions (Gao et al., 2025) on physically valid edits (>=94% vs. 31%). At N=400, cost-tuning lifts planner success from 50.4% to 70.2% and cuts collisions from 19.0% to 8.4%, without domain-specific fine-tuning.","upvotes":1,"discussionId":"6aa28893a2aeb74440b1e068","projectPage":"https://tum-avs.github.io/PlannerForge/","githubRepo":"https://github.com/TUM-AVS/PlannerForge","githubRepoAddedBy":"user","ai_summary":"PlannerForge is an LLM-agent framework that unifies all stages of scenario-based autonomous driving testing and improves generation, selection, modification, and planning performance across commercial and open-source models.","ai_keywords":["LLM-agent framework","scenario-based testing","autonomous driving systems","scenario generation","ADS assessment","cross-attention","prompt conditions","open-source LLMs"],"ai_summary_model":"thinkingmachines/Inkling-Small","githubStars":1,"organization":{"_id":"6a4f83663e43ae629a202b05","name":"TUM-AVS","fullname":"TUM - Professorship of Autonomous Vehicle Systems","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/6723459e7ddd97700c1c1c6a/AJSXS1Qq8ywx4puQ91Fhm.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6a2da6c8ca070ee12c6e396c","avatarUrl":"/avatars/0355287dcabaa67dbc7f0b10b87451f9.svg","isPro":false,"fullname":"Joe Mama","user":"JoeMama123123123","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"organization":{"_id":"6a4f83663e43ae629a202b05","name":"TUM-AVS","fullname":"TUM - Professorship of Autonomous Vehicle Systems","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/6723459e7ddd97700c1c1c6a/AJSXS1Qq8ywx4puQ91Fhm.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2609/2609.08965.md","query":{}}">

Get this paper in your agent:

Don't have the latest CLI?

Cite arxiv.org/abs/2609.08965 in a model README.md to link it from this page.

Cite arxiv.org/abs/2609.08965 in a dataset README.md to link it from this page.

Cite arxiv.org/abs/2609.08965 in a Space README.md to link it from this page.

Add this paper to a collection to link it from this page.