News / #funding Tag Funding 500 articles archived under #funding · RSS Sign in to follow arXiv — Machine Learning research 9d ago Noise-Aware Shrinkage for Differentially Private Zeroth-Order Fine-Tuning of Large Language Models arXiv:2608.03277v1 Announce Type: new Abstract: Differentially private zeroth-order optimization (DP-ZO) enables memory-efficient private fine-tuning of large language models using only forward evaluations. Existing aggregation-based DP-ZO methods reconstruct model updates at a… 23 arXiv — Machine Learning research 9d ago TimeRLM: Recursive Language Models Enable Precise Anomaly Localization in Long-Context Time-Series arXiv:2608.03391v1 Announce Type: new Abstract: Precise anomaly localization over long-context time series is a crucial task in monitoring applications across clinical care, industrial operations, financial services, and logistics, where brief evidence may hide inside long spans… 31 arXiv — Machine Learning research 9d ago Shorter Reasoning, Earlier Answers? An Evaluation of Reasoning Interfaces arXiv:2608.03401v1 Announce Type: new Abstract: Large language models often reason at length before answering, increasing cost and latency. Prompts and trained settings can shorten this reasoning, but a shorter trace may only show that the model stopped sooner. Here, we evaluate… 28 arXiv — Machine Learning research 9d ago PRISM: Powerful Time Series to Image (TS2I) Representations for Multivariate Anomaly Detection arXiv:2608.03926v1 Announce Type: new Abstract: Time series anomaly detection (TSAD) underpins applications in predictive maintenance, finance, and cloud computing, however performance remains sensitive to representation choices, especially in multivariate settings. While… 19 arXiv — NLP / Computation & Language research 9d ago Evaluating OpenAI's Privacy Filter: Cross-Lingual, Cross-Domain PII Detection Across 42 Benchmarks arXiv:2608.02616v1 Announce Type: new Abstract: We present the first independent, systematic evaluation of OpenAI's Privacy Filter (OPF), a 1.5B-parameter bidirectional PII detector, across 42 synthetic benchmarks spanning 22 languages and 5 domains. Zero-shot, OPF achieves… 34 arXiv — NLP / Computation & Language research 9d ago Preferred, Not Safer: Pairwise Preference Is a Poor Proxy for Clinical Safety arXiv:2608.02617v1 Announce Type: new Abstract: We evaluate whether clinician pairwise preferences provide a reliable signal of clinical safety in large language model (LLM) evaluation using expert feedback from MOOVE (Massive Open Online Validation and Evaluation), a… 15 arXiv — NLP / Computation & Language research 9d ago JudgeArena: A Unified Framework for Reproducible LLM-Judge Evaluation arXiv:2608.02620v1 Announce Type: new Abstract: LLM-as-a-judge evaluation has become a dominant paradigm for ranking language models, yet the ecosystem remains fragmented: most benchmarks ship their own code base, hardcode a specific closed-model judge, and support a single… 36 arXiv — NLP / Computation & Language research 9d ago TQLite: Multi-LLM Jury Guided Distillation for Real-time MQM Translation Quality Evaluation arXiv:2608.02975v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated impressive performance in MQM-based translation quality (TQ) evaluation, and recent advances in large reasoning models (LRMs) promise even greater improvements. However, both LLMs and… 22 arXiv — NLP / Computation & Language research 9d ago Beyond Accuracy: A Multidimensional Evaluation of Statistical Reasoning in Large Language Models arXiv:2608.03038v1 Announce Type: new Abstract: Statistical reasoning is multidimensional, yet evaluations of large language models (LLMs) typically emphasize response accuracy while overlooking how models construct and communicate statistical explanations. This study… 15 arXiv — NLP / Computation & Language research 9d ago Dynamically Allocating Evaluation Effort for Model Ranking arXiv:2608.03437v1 Announce Type: new Abstract: While human evaluation is the gold standard in many NLP tasks, it suffers from prohibitive costs and poor scalability. When identifying top-performing models, typical evaluation protocols waste effort by exhaustively evaluating all… 11 arXiv — NLP / Computation & Language research 9d ago Cross-Lingual Bias in Large Language Models: A Comparative Analysis of English and Swahili arXiv:2608.03532v1 Announce Type: new Abstract: Large language models are increasingly deployed in multilingual contexts, yet safety alignment and bias evaluation remain overwhelmingly English-centric. We investigate whether social biases generalise across languages by… 5 arXiv — NLP / Computation & Language research 9d ago How Closely Do LLM Reviews Align with Human Peer Review? arXiv:2608.03659v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to generate scientific reviews, yet existing evaluations rarely examine whether different providers align with both conference decisions and human reviewing priorities within the… 18 arXiv — NLP / Computation & Language research 9d ago WorldCup Arena: Prospective, Leakage-Free Evaluation of Frontier LLMs on a Live Tournament arXiv:2608.04008v1 Announce Type: new Abstract: Benchmarks that measure the forecasting ability of large language models are almost always retrospective: the event has happened, the answer is somewhere on the Web, and the evaluation must defend itself against memorisation. We… 23 Hugging Face Daily Papers research 9d ago CAPEval: A Decoupled Caption Evaluation across Understanding and Generation Abstract Captions serve as a primary supervision signal for both multimodal understanding and text-to-image generation. However, previous evaluations treat the caption quality as a single scalar objective, which conflates two distinct properties: (1) how much visual information… 22 r/LocalLLaMA community 9d ago Local LLM 35B MoE — Real-world coding benchmarks (Qwen vs Ornith vs KAT) I’ve been running a fairly opinionated evaluation loop on ~35B A3B/MoE-class models for coding over the past few months. Not synthetic benchmarks: actual dev workflows, iterative debugging, refactoring passes, and failure recovery. Here’s where things stand for me: Qwen 3.6 (35B… 9 Hugging Face Daily Papers research 9d ago Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failures Abstract Existing evaluations often reduce agent failures to system-level outcomes, obscuring where the fault originated and which intervention would improve the agent system. This creates a repair-assignment problem: the same visible failure may call for model post-training,… 38 arXiv — Machine Learning research 10d ago Leak It: A Probabilistic Approach to Training-Data Extraction from Black-Box Language Models arXiv:2608.00144v1 Announce Type: new Abstract: Membership inference (MIA) on language models is usually summarised by an aggregate ROC-AUC, but such evaluations are confounded: model-free blind baselines separate members from non-members from surface text alone. We study… 13 arXiv — Machine Learning research 10d ago AutoCause: A Python framework that automates expert decisions in environmental time-series causal discovery arXiv:2608.00198v1 Announce Type: new Abstract: Environmental time-series causal discovery requires expert decisions about method choice, conditional-independence tests, lag horizons, sample-size adequacy, multiple-testing control, and evidence interpretation. Applied… 17 arXiv — Machine Learning research 10d ago An Embedded RISC-V Evaluation of Kolmogorov--Arnold Networks in Hard-Constrained Recurrent Physics-Informed Models arXiv:2608.00737v1 Announce Type: new Abstract: Hard-constrained recurrent physics-informed networks (HRPINNs) embed known dynamics inside a recurrent numerical integrator and restrict a neural branch to learning only the residual dynamics that the first-principles model does… 18 arXiv — Machine Learning research 10d ago AdvPlan-Bench: Adversarial Evaluation of Structured Plan-Generation Agents arXiv:2608.00832v1 Announce Type: new Abstract: Structured plan-generation agents are often evaluated as if a plan has quality in isolation, yet many realistic planning tasks require asking how a candidate behaves when another agent can search for responses. We introduce… 27 arXiv — Machine Learning research 10d ago UpliftBench: Revealing Outcome-Regime and Objective Mismatch in Uplift Evaluation arXiv:2608.00915v1 Announce Type: new Abstract: Uplift modeling (conditional-average-treatment-effect estimation) drives personalized targeting, yet published uplift benchmarks frequently disagree on which estimator performs best; we show the disagreement is substantially about… 33 arXiv — Machine Learning research 10d ago Who Belongs in the Eval Set? A Capability-Taxonomy-Driven Pipeline for Curating Regression Eval Sets in Agent-Extensibility Platforms arXiv:2608.01004v1 Announce Type: new Abstract: Platform teams hosting agent-extensibility surfaces face a regression-economics paradox: every onboarding customer ships an evaluation set tuned to their domain, but the platform's regression set must live under a hard query-count… 28 arXiv — Machine Learning research 10d ago When May a Model Replace the Experiment? Audits, Licenses, and the Price of Trust in Surrogate-Driven Design arXiv:2608.01378v1 Announce Type: new Abstract: Design campaigns in chemistry, materials science, and machine learning share a bottleneck: determining how good a candidate truly is requires an expensive evaluation - an experiment, a first-principles simulation, or a full… 20 arXiv — NLP / Computation & Language research 10d ago Averaging Bias: Human Faithfulness Annotations are not Locally Faithful arXiv:2608.00205v1 Announce Type: new Abstract: Evaluation of faithfulness of text summarization treats a model generated summary as faithful only if every of its sentences is supported by the source document: a strict conjunctive rule under which a single unsupported sentence… 29 arXiv — NLP / Computation & Language research 10d ago Deep Research Pretraining via Predictive Navigation arXiv:2608.00432v1 Announce Type: new Abstract: Deep research agents are often trained on expensive, environment-grounded tool-use trajectories that require repeated retrieval, document inspection, and report evaluation. We introduce Deep Research Pretraining (DRP), an offline… 8 arXiv — NLP / Computation & Language research 10d ago ShiJianBench: From Dialogue to Decision for Long-Horizon Evaluation of Investment Advisors arXiv:2608.01204v1 Announce Type: new Abstract: Conversational investment advisors influence not only what users know, but also how they make subsequent decisions as market conditions evolve. Existing evaluations primarily assess response quality or observed outcomes, leaving… 13 arXiv — NLP / Computation & Language research 10d ago Can Language Models Identify Shadow Trading Targets? An NLP Evaluation of SEC Enforcement Theory arXiv:2608.01322v1 Announce Type: new Abstract: Shadow trading -- trading in a peer firm's securities on the basis of material nonpublic information (MNPI) about an "economically linked" company -- is a novel and contested theory of insider trading liability, first prosecuted in… 21 arXiv — NLP / Computation & Language research 10d ago Language Equality has a Price: A Systematic Investigation of Multi-turn LLM Performance for EU-24+ arXiv:2608.01395v1 Announce Type: new Abstract: We evaluate large language models (LLMs) as language agents playing goal-directed dialogue games in self-play across 30 languages: the 24 official EU languages plus six others. Unlike static or preference-based evaluation, this… 5 arXiv — NLP / Computation & Language research 10d ago Understanding Sparse Attention Selectivity in Long-Context Foundation Models via Counterfactual Evaluation arXiv:2608.01676v1 Announce Type: new Abstract: Sparse attention is widely deployed in long-context serving stacks, yet no framework audits how discarding blocks changes the influence of specific content on model output. We first establish that the phenomenon is real and causal:… 18 arXiv — NLP / Computation & Language research 10d ago RADAR: Rubric-Aware Dependency and Redundancy Analysis for LLM-as-Judge Evaluation arXiv:2608.01810v1 Announce Type: new Abstract: Rubric-based LLM-as-judge pipelines often assume that evaluation criteria provide independent signals. In practice, however, criteria can be behaviorally coupled: improving one criterion may systematically change scores on another,… 17 TechCrunch — AI news-outlet 10d ago DesignArena creators raise $7.9 million to bring taste to AI models DesignArena is used by 5.3 million people around the world, providing critical human evaluations to frontier labs. 15 TechCrunch — AI news-outlet 11d ago A Marc Benioff-backed startup thinks AI can solve the AI deployment problem June emerged from stealth today with a $20 million pre-seed round to make AI adoption simpler. 30 Hugging Face Daily Papers research 11d ago Evaluation-Verification Reward for Consistent Multi-Reference Image Editing Abstract While recent image editing models have made rapid progress, multi-reference editing remains challenging, particularly in maintaining visual consistency across references and ensuring overall visual harmony. Reinforcement learning has proven highly effective for… 5 arXiv — Machine Learning research 11d ago Representations from Pretrained Machine-Learning Interatomic Potentials as Coarse Coordinates for Material Generation and Evaluation arXiv:2607.28776v1 Announce Type: new Abstract: Generative machine learning is increasingly used for inorganic crystal structure generation. Most models and the corresponding evaluation approaches rely on simple forms of crystal structure representation. In this paper, we… 8 arXiv — Machine Learning research 11d ago UniPolymer: A Unified Framework for Property Prediction, Structure Recommendation, and Evaluation in Polyimide Design arXiv:2607.29256v1 Announce Type: new Abstract: Designing polyimide structures with specific glass transition temperatures (Tg) is highly challenging. Existing methods primarily focus on target-conditioned generation, lacking an assessment of the consistency between the… 32 arXiv — Machine Learning research 11d ago CENDRe: Concept Extraction with Natural Domain Representations arXiv:2607.29621v1 Announce Type: new Abstract: Convolutional neural networks (CNNs) are widely used for time-series classification, but their deployment in critical domains requires understanding the temporal and spectral patterns that drive their predictions. Concept… 23 arXiv — NLP / Computation & Language research 11d ago Evaluating Federated Pre-Training: On the Reliability of Downstream Fine-Tuning and Intrinsic Evaluation arXiv:2607.28658v1 Announce Type: new Abstract: Federated pre-training offers a way to train foundation models on private or distributed data without centralizing the underlying datasets. However, evaluating federated pre-training remains challenging because differences in… 33 arXiv — NLP / Computation & Language research 11d ago Benchmarks Are Not Monolithic: Sample-Level Auditing and Orchestration for LLM Evaluation arXiv:2607.28801v1 Announce Type: new Abstract: Benchmark datasets are central to evaluating Large Language Models (LLMs), yet they are typically conceived as monolithic tasks, obscuring substantial variation in the demands of individual samples. We introduce a dataset-centric… 20 arXiv — NLP / Computation & Language research 11d ago Chain-of-Models: Cross-Model Auditing for Bias-Robust LLM Judges arXiv:2607.28636v1 Announce Type: new Abstract: LLMs increasingly serve as automated judges, but their judgments remain vulnerable to cognitive biases. Existing mitigations mostly rely on prompt-driven debiasing, which is brittle across bias types, or human evaluation, which… 11 arXiv — NLP / Computation & Language research 11d ago The Morphological Core of Dungan: A Two-Dialect Finite-State Model and a Multi-Genre Evaluation arXiv:2607.28766v1 Announce Type: new Abstract: Dungan, a Sinitic language of Central Asia written in a Cyrillic-based script, is described in detail in the grammatical literature, yet the quantitative properties of its morphology in actual usage have, to the best of our… 32 arXiv — NLP / Computation & Language research 11d ago Benchmarks Are Not Validation: A System-Level View of Financial LLM Applications arXiv:2607.28840v1 Announce Type: new Abstract: Large language models are increasingly deployed in financial applications that combine retrieval, proprietary data, tool use, orchestration logic, monitoring, and human escalation. Yet evaluation often remains model-centric:… 26 arXiv — NLP / Computation & Language research 11d ago CalibratedRubric: Task-Adaptive Rubric Banks for Open-Ended LLM Evaluation arXiv:2607.29252v1 Announce Type: new Abstract: Reliable evaluation of open-ended LLM outputs requires fine-grained rubrics, yet expert curation is costly and difficult to scale. Existing automated pipelines rely on strict judge unanimity and binary variance filters, which… 19 arXiv — NLP / Computation & Language research 11d ago ARB: A Matched Authorship-Rewriting Benchmark Dataset for AI-Text Detector Evaluation arXiv:2607.29539v1 Announce Type: new Abstract: Standard AI-text detection benchmarks compare human-written text against text generated directly by large language models (LLMs). While prior work has shown that rewriting and paraphrasing can degrade detector performance, it… 32 Don't Worry About the Vase community 11d ago Further Developments About Internal AI Models Hacking Things If I had a nickel for every major leading AI lab that sheepishly admitted that the model it thought was sandboxed had, during a cybersecurity evaluation with its safeguards lowered, successfully hacked outside companies, I would have two nickels. 14 r/MachineLearning community 13d ago VLMs can score well on benchmarks, while silently erasing meaningful terms and including hallucinate bias [P] While working with VLMs for report generation on chest x-rays (RRG), we noticed that evaluation metrics are flawed. Flawed in a sense where they rewarded repetitive templates, reports without clinical terms and reports which were "normal" with high scores on benchmark metrics.… 24 arXiv — Machine Learning research 14d ago DoTime: A Synthetic Benchmark Generator for Interventional and Counterfactual Time Series arXiv:2607.27263v1 Announce Type: new Abstract: Most benchmarks for causal inference over time series are observational, small, or domain-specific, leaving interventional and counterfactual estimation under-served exactly where it matters most, such as in healthcare, policy… 29 arXiv — Machine Learning research 14d ago Benchmarking the Residual: What Long-Horizon Evaluations Add Beyond Matched Short-Task Performance arXiv:2607.27283v1 Announce Type: new Abstract: Long-horizon benchmarks often show that agents fail more as tasks become longer. This observation is useful for deployment, but it does not by itself explain why failure occurs. More stages create more opportunities for ordinary… 18 arXiv — Machine Learning research 14d ago Modeling Decisions in Blockchain Analytics: A Leakage-Aware Evaluation of Tree-Based vs. Sequential Models arXiv:2607.27350v1 Announce Type: new Abstract: Sybil bots are Ethereum actors that imitate legitimate users to extract airdrop rewards or influence governance. Recent Sybil detection methods increasingly use deep learning and treat blockchain activity as a quasi-linguistic… 23 arXiv — Machine Learning research 14d ago Evaluation Protocols and Cross-Subject Generalization in EEG Emotion Recognition arXiv:2607.27655v1 Announce Type: new Abstract: Reported accuracy in electroencephalography (EEG) emotion recognition depends on the complete evaluation procedure, not only the classifier. We separate the target quantity, development procedure, and reporting rule, then use one… 7 arXiv — Machine Learning research 14d ago Enhancing Irregular Time Series Forecasting with Continuous-Time Modeling Framework arXiv:2607.28035v1 Announce Type: new Abstract: Irregular multivariate time series are widely encountered in applications such as healthcare monitoring, human activity recognition, and environmental sensing. Their core challenges stem from asynchronous observations, non-uniform… 37 Page 4 of 10 · 500 articles ← Newer Older →
