Agent safety shouldn’t stop at training: our new paper argues for runtime contracts that block dangerous actions and require checkable evidence that agents actually did what they claim.</p>\n","updatedAt":"2026-08-13T01:59:28.581Z","author":{"_id":"62b32a4429a410b7f6b06710","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/62b32a4429a410b7f6b06710/VzgvmnlYZWuifZTkIkCxy.jpeg","fullname":"Wenhao Wang","name":"WenhaoWang","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":21,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.9198559522628784},"editors":["WenhaoWang"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/62b32a4429a410b7f6b06710/VzgvmnlYZWuifZTkIkCxy.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2608.11274","authors":[{"_id":"6a7d24b40ac8bee77474ee2c","name":"Albus W. Ng","hidden":false},{"_id":"6a7d24b40ac8bee77474ee2d","name":"Yi Han","hidden":false},{"_id":"6a7d24b40ac8bee77474ee2e","name":"Jusheng Zhang","hidden":false},{"_id":"6a7d24b40ac8bee77474ee2f","name":"Wenhao Wang","hidden":false}],"publishedAt":"2026-08-11T00:00:00.000Z","submittedOnDailyAt":"2026-08-13T00:00:00.000Z","title":"Agent Safety Should Be a Runtime Contract","submittedOnDailyBy":{"_id":"62b32a4429a410b7f6b06710","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/62b32a4429a410b7f6b06710/VzgvmnlYZWuifZTkIkCxy.jpeg","isPro":false,"fullname":"Wenhao Wang","user":"WenhaoWang","type":"user","name":"WenhaoWang"},"summary":"The dominant paradigm treats AI safety as a property to be instilled during model training via RLHF, DPO, or Constitutional AI. We argue this is structurally insufficient for autonomous agents that execute code, mutate files, send messages, and modify databases. Agent safety should be a runtime contract enforced by the harness, and the contract has two complementary faces. The preventive face blocks dangerous actions before they happen via sandboxes, permission gates, output filters, and trajectory monitors. The evidential face requires verifiable proof that good actions actually happened, gating task submission on hard evidence such as test runs, log captures, file diffs, and citation grounding. We ground the position in four lines of public evidence, with row-level protocols and data released in the supplementary JSON files: a survey of 52 documented AI-agent and LLM safety incidents, a false-completion audit with 31 non-contested core cases plus one disputed illustrative case, a trajectory-schema audit of 12 public agent systems and harnesses, and a title-level audit of all 28,560 papers accepted at NeurIPS, ICML, and ICLR 2023-2025 showing a pooled 8-12x imbalance between training-time and deployment-time publication. Two prior communities that needed to enforce safety, computer security and the experimental sciences, converged on runtime contracts with both preventive and evidential elements; agentic AI is now under the same pressure. We formalize an Agent Trajectory Schema and Evidence Chain, state a compositional gating proposition based on standard monitor composition, and outline a research agenda. The right unit of safety in agentic AI is the trajectory-with-checkable-evidence, not the model.","upvotes":3,"discussionId":"6a7d24b50ac8bee77474ee30","ai_summary":"Agent safety should be enforced at runtime through preventive controls and verifiable evidence rather than relying solely on training-time alignment methods.","ai_keywords":["RLHF","DPO","Constitutional AI","sandboxes","permission gates","output filters","trajectory monitors","file diffs","citation grounding","trajectory-schema audit","Agent Trajectory Schema","Evidence Chain","compositional gating","monitor composition"],"ai_summary_model":"thinkingmachines/Inkling-Small"},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"67222d6797102382cbbd387c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/mtD0mApQTOOWowBtTozRR.png","isPro":false,"fullname":"TIP-I2V","user":"tipi2v","type":"user"},{"_id":"6a2f82dc43f2af6bf517cc37","avatarUrl":"/avatars/82ae9e90e55f6c839ac93add52215e85.svg","isPro":false,"fullname":"Lab","user":"Sydney-Vast","type":"user"},{"_id":"63ac5701c21e60a3e9b58aa7","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/63ac5701c21e60a3e9b58aa7/g6EX7diOpuA94R2ab-rZC.png","isPro":true,"fullname":"Dipankar Sarkar","user":"dipankarsarkar","type":"user"}],"acceptLanguages":["en"],"dailyPaperRank":0,"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2608/2608.11274.md","query":{}}">
Agent Safety Should Be a Runtime Contract
Abstract
Agent safety should be enforced at runtime through preventive controls and verifiable evidence rather than relying solely on training-time alignment methods.
The dominant paradigm treats AI safety as a property to be instilled during model training via RLHF, DPO, or Constitutional AI. We argue this is structurally insufficient for autonomous agents that execute code, mutate files, send messages, and modify databases. Agent safety should be a runtime contract enforced by the harness, and the contract has two complementary faces. The preventive face blocks dangerous actions before they happen via sandboxes, permission gates, output filters, and trajectory monitors. The evidential face requires verifiable proof that good actions actually happened, gating task submission on hard evidence such as test runs, log captures, file diffs, and citation grounding. We ground the position in four lines of public evidence, with row-level protocols and data released in the supplementary JSON files: a survey of 52 documented AI-agent and LLM safety incidents, a false-completion audit with 31 non-contested core cases plus one disputed illustrative case, a trajectory-schema audit of 12 public agent systems and harnesses, and a title-level audit of all 28,560 papers accepted at NeurIPS, ICML, and ICLR 2023-2025 showing a pooled 8-12x imbalance between training-time and deployment-time publication. Two prior communities that needed to enforce safety, computer security and the experimental sciences, converged on runtime contracts with both preventive and evidential elements; agentic AI is now under the same pressure. We formalize an Agent Trajectory Schema and Evidence Chain, state a compositional gating proposition based on standard monitor composition, and outline a research agenda. The right unit of safety in agentic AI is the trajectory-with-checkable-evidence, not the model.
Community
Agent safety shouldn’t stop at training: our new paper argues for runtime contracts that block dangerous actions and require checkable evidence that agents actually did what they claim.
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Cite arxiv.org/abs/2608.11274 in a model README.md to link it from this page.
Cite arxiv.org/abs/2608.11274 in a dataset README.md to link it from this page.
Cite arxiv.org/abs/2608.11274 in a Space README.md to link it from this page.
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.