News / #safety Tag Safety + alignment 500 articles archived under #safety · RSS Sign in to follow Hacker News — AI on Front Page community 1mo ago Opening up 'Zero-Knowledge Proof' technology to promote privacy in age assurance Article URL: https://blog.google/innovation-and-ai/technology/safety-security/opening-up-zero-knowledge-proof-technology-to-promote-privacy-in-age-assurance/ Comments URL: https://news.ycombinator.com/item?id=48753979 Points: 200 # Comments: 212 34 Ars Technica — AI news-outlet 1mo ago After spooking Trump into safety testing, Anthropic AI models get global release US lifts curbs on Anthropic’s advanced Fable and Mythos models. 31 llama.cpp releases dev-tools 1mo ago b9857 hexagon: flash attention rework (optimizations, accuracy improvements, etc) ( #25085 ) hex-mm: fold mm quant tasks into the main matmul threads hex-mm: minor formatting fixes hex-mm: cleanup is_quant checks in dma dispatch hex-mm: fix dst-spad alignment hex-mm: move fp kernels… 5 r/MachineLearning community 1mo ago A system-level approach to prompt injection: separating instruction and data channels in LLM agents [P] Prompt injection has emerged as one of the most persistent failure modes in tool-using LLM systems, particularly in agentic workflows where models interact with external data sources. Most mitigation strategies focus on input filtering or model-side alignment, but these… 9 Hugging Face Daily Papers research 1mo ago RedVox: Safety and Fairness Gaps in Speech Models Across Languages Abstract Multilingual safety and fairness benchmark for speech models reveals persistent vulnerabilities across languages and naturalistic conditions. Generated by Qwen/Qwen2.5-Coder-32B-Instruct Speech-capable models are increasingly deployed in real-world applications across… 36 Vercel — AI dev-tools 1mo ago Claude Fable 5 access restored on AI Gateway Access to Claude Fable 5, the Mythos-class model, has now been restored on AI Gateway following the US Government's decision to lift the export controls. Fable 5 is the same model that was available between June 9 and June 12. What has changed is the safety classifiers, which… 27 Hugging Face Daily Papers research 1mo ago GEAR: Guided End-to-End AutoRegression for Image Synthesis Abstract GEAR trains a vector-quantized tokenizer and autoregressive generator jointly end-to-end using representation alignment, overcoming non-differentiability issues through a dual read-out approach that improves convergence speed and feature quality. Generated by… 36 arXiv — NLP / Computation & Language research 1mo ago Revocable Learned State via Process Sidecars arXiv:2606.30788v1 Announce Type: cross Abstract: Language models are often adapted in stages: a public skill phase, a private memory phase, and a later safety phase that learns to refuse outputs tied to the remembered entities. Revoking the memory after the safety phase is not… 17 arXiv — Machine Learning research 1mo ago Safe Online Learning via Smooth Safety-Structured Policy Composition arXiv:2606.31320v1 Announce Type: new Abstract: Safe online reinforcement learning requires policies to respect safety constraints while maintaining smooth optimization dynamics. Existing approaches typically rely on either strict safety enforcement via action interventions,… 7 arXiv — Machine Learning research 1mo ago Resolving superposition in AI for interpretability and cross-modal alignment in patient-neuronal images arXiv:2606.31394v1 Announce Type: new Abstract: Artificial intelligence is transforming our capability to solve biological challenges. In dimensionality bottleneck regimes exacerbated by high-dimensional biological data, Neural networks force distinct concepts into the lower… 14 arXiv — Machine Learning research 1mo ago On the Convergence of Self-Improving Online LLM Alignment arXiv:2606.31524v1 Announce Type: new Abstract: The Self-Improving Alignment (SAIL) algorithm addresses distribution shift by reducing a bilevel formulation of the problem to an efficient, single-level method. Empirically, SAIL has demonstrated strong performance on this task.… 8 arXiv — Machine Learning research 1mo ago Evil Spectra: How Optimisers can Amplify or Suppress Emergent Misalignment arXiv:2606.31591v1 Announce Type: new Abstract: Emergent misalignment (EM) is a recently discovered phenomenon in LLMs where fine-tuning on a narrow misaligned task, such as writing insecure code, leads to broadly misaligned behaviour on unrelated prompts. Previous work has… 11 arXiv — Machine Learning research 1mo ago Addressing Over-Refusal in LLMs with Competing Rewards arXiv:2606.31748v1 Announce Type: new Abstract: Safety training on language models often induces over-refusal: improved safety on harmful prompts at the cost of increased refusal on harmless ones. Though this trade-off can be mitigated by training models with reinforcement… 25 arXiv — NLP / Computation & Language research 1mo ago Signed-Permutation Coordinate Transport for RMSNorm Transformers arXiv:2606.31963v1 Announce Type: cross Abstract: Modern LLM workflows move coordinate-indexed objects across checkpoints: steering vectors, sparse autoencoders, top-$k$ neuron sets, attribution lists, and merge alignments. This is only well posed after fixing the model's… 37 arXiv — NLP / Computation & Language research 1mo ago From Propositional to Perceptual Asymmetry: Extending Frictive Policy Optimization to Asymmetric Partial Information Dialogue arXiv:2606.30973v1 Announce Type: new Abstract: Frictive Policy Optimization (FPO; Pustejovsky et al., 2025) treats friction in collaborative dialogue -- misalignment, misunderstanding, repair -- as an epistemic signal essential to common-ground construction, rather than noise… 18 arXiv — NLP / Computation & Language research 1mo ago LOPA: Enhancing Spoken Language Assessment via Latent Ordinal Prototype Alignment arXiv:2606.31310v1 Announce Type: new Abstract: Fueled by increasing model scale and multimodal inputs, Multimodal Large Language Models (MLLMs) have emerged as a promising paradigm for Spoken Language Assessment (SLA). While effective, this paradigm often overlooks the… 9 arXiv — NLP / Computation & Language research 1mo ago Moral Safety in LLMs: Exposing Performative Compliance with Puzzled Cues arXiv:2606.31644v1 Announce Type: new Abstract: As large language models take on morally consequential roles in healthcare, legal, and hiring contexts, we need to examine whether their ethical behaviors are genuine or superficial. We show that current fairness evaluations… 5 arXiv — NLP / Computation & Language research 1mo ago ADAPT: Attention Dynamics Alignment with Preference Tuning for Faithful MLLMs arXiv:2606.31054v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) are critically hampered by hallucination, generating content inconsistent with the provided image. In this paper, we identify an internal signature of hallucination: progressive… 37 TechCrunch — AI news-outlet 1mo ago Anthropic launches Claude Sonnet 5 as a cheaper way to run agents Anthropic’s Claude Sonnet 5 brings stronger agentic capabilities, lower pricing, and improved safety, positioning the model as a cheaper alternative to Opus, GPT-5.5, and Gemini Pro. 5 Hugging Face Daily Papers research 1mo ago Mind the Heads: Topological Representation Alignment for Multimodal LLMs Abstract HeRA aligns individual attention heads in MLLMs to preserve local neighborhood relationships across modalities, improving vision-centric task performance and reducing visual hallucinations. Generated by Qwen/Qwen2.5-Coder-32B-Instruct Representation alignment has… 27 Hugging Face Daily Papers research 1mo ago A Gravitational Interpretation of Fine-Tuning Reversion Abstract Post-alignment safety degradation arises from geometric properties of training history, where fine-tuning reversion follows a persistent direction defined by early training dynamics. Generated by Qwen/Qwen2.5-Coder-32B-Instruct Fine-tuning on harmless data can partially… 35 arXiv — Machine Learning research 1mo ago A Gravitational Interpretation of Fine-Tuning Reversion arXiv:2606.28525v1 Announce Type: new Abstract: Fine-tuning on harmless data can partially undo behaviors acquired earlier in training. Safety can erode under benign post-alignment updates, unlearned capabilities can re-emerge, latent traits can transfer through apparently… 27 arXiv — Machine Learning research 1mo ago MOSAIC: Orchestrating Collaborative Knowledge Tracing with Hierarchical Semantic Alignment arXiv:2606.29049v1 Announce Type: new Abstract: Knowledge Tracing (KT) is important for personalized education but traditionally suffers from two key limitations: a reliance on shallow ID-based representations that neglect semantic depth and a restriction to single-granularity… 37 arXiv — Machine Learning research 1mo ago Representational Depth of Evaluation Awareness Shifts With Scale in Open-Weight Language Models arXiv:2606.29196v1 Announce Type: new Abstract: Do language models know when they are being tested? This question matters for AI safety: a model that recognises an evaluation context could alter its behaviour strategically, making downstream benchmarks harder to interpret. Using… 27 arXiv — Machine Learning research 1mo ago Beyond Trajectory Matching: Reflow with Marginal Distribution Alignment arXiv:2606.29287v1 Announce Type: new Abstract: Diffusion and continuous-flow generative models achieve high-quality generation, and their deterministic sampling can be formulated as solving learned ODE dynamics. However, accurate ODE discretization often requires many steps,… 36 arXiv — Machine Learning research 1mo ago Do Models Read What They Write? Causal Registers in Scratchpad Reasoning arXiv:2606.29522v1 Announce Type: new Abstract: A central hope behind process supervision is that models can expose intermediate variables that matter for their later behavior. For this to help with alignment, a scratchpad must be tied to the computation: when the model writes a… 29 arXiv — Machine Learning research 1mo ago VISTA-DZ: Visual Semantic Trajectory Adaptation for Personalized Dilemma Zone Prediction arXiv:2606.29548v1 Announce Type: new Abstract: Driver decision making in the dilemma zone at signalized intersections is safety critical, as vehicles approaching a yellow signal must decide whether to stop or proceed within limited time and distance margins. Accurate prediction… 38 arXiv — NLP / Computation & Language research 1mo ago DriftGuard: Safety-Aware Multi-Monitor Detection and Selective Adaptation for Evolving Toxicity Moderation arXiv:2606.28725v1 Announce Type: new Abstract: Automated toxicity moderation systems operate in dynamic online environments where harmful behavior evolves through coded language, shifting targets, and strategic adaptation to enforcement. Existing drift detection methods often… 12 arXiv — NLP / Computation & Language research 1mo ago The Heterogeneous Safety Impacts of Benign Multilingual Fine-Tuning arXiv:2606.28843v1 Announce Type: new Abstract: Fine-tuning a large language model is a ubiquitous method for enhancing its capability on a specific downstream task. However, prior work has shown that this increase in capability comes with a cost: it can increase a model's… 18 arXiv — NLP / Computation & Language research 1mo ago A Hybrid Framework for Song Lyric Annotation Based on Human-LLM Alignment arXiv:2606.29273v1 Announce Type: new Abstract: Emotion recognition of song lyrics is a challenging task since lyrics may not necessarily align with the overall emotion of a song. As a result, lyrics annotation remains largely underexplored. Drawing inspiration from research in… 34 arXiv — NLP / Computation & Language research 1mo ago Resolution Thresholds in VLM Detection of Harmful ASCII Art Across Construction Modes and Languages arXiv:2606.29649v1 Announce Type: new Abstract: Large Vision-Language Models (VLMs) are increasingly deployed as content moderation tools, yet they remain vulnerable to jailbreak attacks in which harmful text is visually encoded as ASCII art. This can allow inappropriate or… 31 arXiv — NLP / Computation & Language research 1mo ago Timesteps of Mamba Align with Human Reading Times arXiv:2606.29904v1 Announce Type: new Abstract: This study demonstrates an alignment of per-word processing time in a popular state-space language model Mamba and human readers. In Mamba, the recurrent state transition at each layer conceptually takes some duration of time, the… 12 arXiv — NLP / Computation & Language research 1mo ago Towards Physical Intuitions for Alignment Dynamics: A Case Study With Randomness Crystallization arXiv:2606.29933v1 Announce Type: new Abstract: The alignment of language models is typically studied through the lens of capability benchmarks, but the dynamics of how models change during post-training remain poorly understood. We argue that the physical sciences, and… 16 arXiv — NLP / Computation & Language research 1mo ago Node-to-Neighborhood Semantic Consistency: Text-Topology Alignment for TAGs Anomaly Detection arXiv:2606.30009v1 Announce Type: new Abstract: Graph anomaly detection (GAD) on text-attributed graphs (TAGs) is vital for applications such as fraud detection and academic integrity verification. Existing approaches generally fall into two paradigms. GNN-based methods… 36 Hugging Face Daily Papers research 1mo ago SafePyramid: A Hierarchical Benchmark for In-context Policy Guardrailing Abstract SafePyramid benchmark evaluates guardrail systems' ability to identify safety violations through in-context policy specification across multiple domains and complexity levels. Generated by Qwen/Qwen2.5-Coder-32B-Instruct In real-world applications, guardrails are often… 5 arXiv — Machine Learning research 1mo ago RS-Diffuser: Risk-Sensitive Diffusion Planning with Distributional Value Guidance arXiv:2606.27766v1 Announce Type: new Abstract: Offline reinforcement learning enables policy learning from fixed datasets without additional environment interaction, making it appealing for safety-critical applications where online exploration is costly or unsafe.… 32 arXiv — Machine Learning research 1mo ago NormGuard: Reward-Preserving Norm Constraints in Flow-Matching Reinforcement Learning arXiv:2606.27771v1 Announce Type: new Abstract: Reinforcement learning (RL) post-training improves the reward alignment of flow-based generators, but often degrades perceptual quality in ways that are not captured by the reward proxy. We identify a simple structural signature of… 8 arXiv — Machine Learning research 1mo ago OperatorSHAP: Fast and Accurate Shapley Value Estimation for Neural Operators arXiv:2606.28065v1 Announce Type: new Abstract: Understanding model predictions is essential for physical applications, where outputs often inform safety-critical decisions, such as structural load assessment, weather warnings, and clinical diagnosis. Shapley values satisfy many… 20 arXiv — Machine Learning research 1mo ago Democratic ICAI: Debating Our Way to Steering Principles from Preferences arXiv:2606.28294v1 Announce Type: new Abstract: Preference-based alignment often struggles to capture the reasoning that underlies human judgments. Many evaluations rely on multiple interacting criteria, yet pairwise labels reveal only the final choice rather than the… 38 arXiv — NLP / Computation & Language research 1mo ago Position: The Term "Machine Unlearning" Is Overused in LLMs arXiv:2606.27379v1 Announce Type: new Abstract: Large language models increasingly face demands to "forget" training data, knowledge, or behaviors due to regulatory deletion obligations, copyright/licensing disputes, and safety or product-policy requirements. This position paper… 15 arXiv — Machine Learning research 1mo ago Physics-Guided Robotic Radiation Source Localization along Arbitrary Measurement Paths in Unstructured Environments arXiv:2606.27624v1 Announce Type: cross Abstract: Using robots to estimate the location of the radiation source is an effective way to improve efficiency and safety. Existing methods focus on planning the robot's path to achieve precise estimation, typically approaching the… 19 arXiv — NLP / Computation & Language research 1mo ago Yuvion LLM: An Adversarially-Aware Large Language Model for Content And AI Safety arXiv:2606.27632v1 Announce Type: new Abstract: As large language models are increasingly deployed in real-world systems, safety failures can still lead to harmful outputs and dangerous misuse. We argue that the essence of safety is adversarial: many failures arise not from… 29 arXiv — NLP / Computation & Language research 1mo ago Enhancing Numerical Prediction in LLMs via Smooth MMD Alignment arXiv:2606.27731v1 Announce Type: new Abstract: Despite their strong general capabilities, large language models (LLMs) often remain unreliable when outputs must be numerically precise. A key reason is the training objective: standard cross-entropy treats numeric tokens as… 31 arXiv — NLP / Computation & Language research 1mo ago Check Yourself Before You Wreck Yourself: Selectively Quitting Improves LLM Agent Safety arXiv:2510.16492v4 Announce Type: replace Abstract: As Large Language Model (LLM) agents increasingly operate in complex environments with real-world consequences, their safety becomes critical. While uncertainty quantification is well-studied for single-turn tasks, multi-turn… 20 arXiv — NLP / Computation & Language research 1mo ago SingGuard: A Policy-Adaptive Multimodal LLM Guardrail with Dynamic Reasoning arXiv:2606.22873v3 Announce Type: replace-cross Abstract: Vision-language models (VLMs) are increasingly deployed in consumer, medical, financial, and enterprise applications. This broad deployment expands the safety surface: risks can arise from multimodal question answering,… 31 r/LocalLLaMA community 1mo ago [NEW MODEL] - SupraSafety-18M · Tiny Content-Moderation Model Hey r/LocalLLaMA ! SupraLabs is back with a new model: SupraSafety-18M . It's a BERT-style 18M params model trained from scratch on 2 T4 GPUs in Kaggle on the nvidia/Nemotron-3.5-Content-Safety-Dataset dataset for 7 epochs. It's built to run on edge devices , mobile phones , or… 13 Hugging Face Daily Papers research 1mo ago LISA: Likelihood Score Alignment for Visual-condition Controllable Generation Abstract Score-based generative modeling reveals that side networks contribute likelihood scores to conditional control, leading to improved training efficiency through likelihood score alignment regularization. Generated by Qwen/Qwen2.5-Coder-32B-Instruct The prevalent… 36 OpenAI official-blog 1mo ago Previewing GPT-5.6 Sol: a next-generation model OpenAI previews GPT-5.6 Sol, a next-generation model with stronger capabilities in coding, science, and cybersecurity, paired with its most advanced safety stack. 10 Smol AI News news-outlet 1mo ago not much happened today **OpenAI** previewed **GPT-5.6** with three variants: **Sol** (flagship), **Terra** (mid-tier), and **Luna** (lower-cost), launching under a restricted rollout mandated by the U.S. government, limiting access to trusted partners. **Sol** boasts enhanced cybersecurity and safety… 35 arXiv — Machine Learning research 1mo ago Necessary but Not Sufficient: Temperature Control and Reproducibility in LLM-as-Judge Safety Evaluations arXiv:2606.26185v1 Announce Type: new Abstract: LLM-as-judge ("grader") components are now standard in evaluation harnesses, including safety evaluations where a pass/fail verdict may gate downstream deployment decisions. A widespread assumption is that setting the grader's… 4 Page 9 of 10 · 500 articles ← Newer Older →
