Data Harness for automatic LLM data preparation pipelines</p>\n","updatedAt":"2026-07-22T03:27:07.618Z","author":{"_id":"6751a4fedf636b0140a9b873","avatarUrl":"/avatars/d75f7f6cfbfb4d646e0e557d1cfacdce.svg","fullname":"Hao Liang","name":"lhpku20010120","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":4,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.6313618421554565},"editors":["lhpku20010120"],"editorAvatarUrls":["/avatars/d75f7f6cfbfb4d646e0e557d1cfacdce.svg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2607.16617","authors":[{"_id":"6a5f12b34fe5d1d13e84aca9","user":{"_id":"65b7098af327f1f4e315294d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65b7098af327f1f4e315294d/38JtHebJ_XYWciMA15vJ3.jpeg","isPro":false,"fullname":"Runming He","user":"blackBOX25I47","type":"user","name":"blackBOX25I47"},"name":"Runming He","status":"claimed_verified","statusLastChangedAt":"2026-07-22T07:39:29.236Z","hidden":false},{"_id":"6a5f12b34fe5d1d13e84acaa","name":"Zhen Hao Wong","hidden":false},{"_id":"6a5f12b34fe5d1d13e84acab","name":"Hao Liang","hidden":false},{"_id":"6a5f12b34fe5d1d13e84acac","name":"Zimo Meng","hidden":false},{"_id":"6a5f12b34fe5d1d13e84acad","name":"Chengyu Shen","hidden":false},{"_id":"6a5f12b34fe5d1d13e84acae","name":"Xiaochen Ma","hidden":false},{"_id":"6a5f12b34fe5d1d13e84acaf","name":"Wentao Zhang","hidden":false}],"publishedAt":"2026-07-18T00:00:00.000Z","submittedOnDailyAt":"2026-07-22T00:00:00.000Z","title":"DataFlow-Harness: A Grounded Code-Agent Platform for Constructing Editable LLM Data Pipelines","submittedOnDailyBy":{"_id":"6751a4fedf636b0140a9b873","avatarUrl":"/avatars/d75f7f6cfbfb4d646e0e557d1cfacdce.svg","isPro":false,"fullname":"Hao Liang","user":"lhpku20010120","type":"user","name":"lhpku20010120"},"summary":"Large language models (LLMs) are increasingly used to automate data-processing workflows, yet coding agents typically produce scripts that are not automatically materialized as persistent, editable platform artifacts. We call this disconnect the NL2Pipeline gap. To bridge it, we introduce DataFlow-Harness, a platform that guides an LLM agent to construct platform-native directed acyclic graphs (DAGs) through typed, incremental mutations rather than free-form scripts. The platform combines DataFlow-Skills for procedural guidance, a Model Context Protocol (MCP) layer that exposes the live operator registry and current pipeline state, and DataFlow-WebUI, which synchronizes conversational authoring with a visual DAG editor. On a 12-task data-engineering benchmark, DataFlow-Harness achieves a 93.3\\% observed end-to-end pass rate. Relative to Vanilla Claude Code, it reduces measured monetary cost by 72.5\\% and generation latency by 49.9\\%; its observed pass rate is within 0.9 percentage points of the Context-Aware Claude Code baseline while its cost is 42.8\\% lower. Per-task analysis indicates that Skills are most useful when construction depends on implicit procedural knowledge. These results show that live platform grounding can produce persistent, editable workflow artifacts with an observed reliability close to script-generation baselines and with lower measured construction cost and latency.","upvotes":51,"discussionId":"6a5f12b34fe5d1d13e84acb0","organization":{"_id":"61dcd8e344f59573371b5cb6","name":"PekingUniversity","fullname":"Peking University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/vavgrBsnkSejriUF4lXDE.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6751a4fedf636b0140a9b873","avatarUrl":"/avatars/d75f7f6cfbfb4d646e0e557d1cfacdce.svg","isPro":false,"fullname":"Hao Liang","user":"lhpku20010120","type":"user"},{"_id":"6890911567be93a3695bee7e","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/mik3yDr_AqK-THkLKl3Zp.png","isPro":false,"fullname":"Juno Xu","user":"Junooooo","type":"user"},{"_id":"63ac5701c21e60a3e9b58aa7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63ac5701c21e60a3e9b58aa7/g6EX7diOpuA94R2ab-rZC.png","isPro":true,"fullname":"Dipankar Sarkar","user":"dipankarsarkar","type":"user"},{"_id":"6217599529500f41901123f8","avatarUrl":"/avatars/8a0fe54e53fe6527c70a78598a0cd941.svg","isPro":false,"fullname":"Hao Liang","user":"lhbit20010120","type":"user"},{"_id":"66ac9567c97d2f0c88c3ac72","avatarUrl":"/avatars/14df8b5eed4ea756c93f61999c75e44f.svg","isPro":false,"fullname":"PKU_Baichuan","user":"PKU-Baichuan","type":"user"},{"_id":"65b7098af327f1f4e315294d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65b7098af327f1f4e315294d/38JtHebJ_XYWciMA15vJ3.jpeg","isPro":false,"fullname":"Runming He","user":"blackBOX25I47","type":"user"},{"_id":"6618a60721d5003025004c96","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6618a60721d5003025004c96/_9DZG3lKbIt5KOj4WxsIJ.jpeg","isPro":false,"fullname":"Meiyi Qiang","user":"MeiyiQiang","type":"user"},{"_id":"6880e7df575954f53124c68c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/NG-VcF3FvLUxmVkAUTu9w.png","isPro":false,"fullname":"Qifeng Xia","user":"PiarPP","type":"user"},{"_id":"67c06f81562f69fd3dce231e","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/OHQoUdh_lhBfvGEmAVVPV.png","isPro":false,"fullname":"Keesh","user":"Miaode","type":"user"},{"_id":"620783f24e28382272337ba4","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/620783f24e28382272337ba4/zkUveQPNiDfYjgGhuFErj.jpeg","isPro":false,"fullname":"GuoLiangTang","user":"Tommy930","type":"user"},{"_id":"668cfdacf278aa900c8e00be","avatarUrl":"/avatars/0c03d223736448480dd434d54ada249e.svg","isPro":false,"fullname":"Hengyi Feng","user":"Heinz217","type":"user"},{"_id":"68e28ee3f0f3fb8ef0c62af9","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/68e28ee3f0f3fb8ef0c62af9/vttjWCAjd7cX_nA8pyZoE.jpeg","isPro":false,"fullname":"Fish","user":"Fishiing","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":1,"organization":{"_id":"61dcd8e344f59573371b5cb6","name":"PekingUniversity","fullname":"Peking University","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/vavgrBsnkSejriUF4lXDE.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2607/2607.16617.md","query":{}}">

Get this paper in your agent:

Don't have the latest CLI?

Cite arxiv.org/abs/2607.16617 in a model README.md to link it from this page.

Cite arxiv.org/abs/2607.16617 in a dataset README.md to link it from this page.

Cite arxiv.org/abs/2607.16617 in a Space README.md to link it from this page.

Add this paper to a collection to link it from this page.