News / #training Tag Training 422 articles archived under #training · RSS Sign in to follow arXiv — Machine Learning research 28d ago From Demonstrations to Rewards: Test-Time Prompt Optimization for VLM Reward Models arXiv:2606.00083v1 Announce Type: new Abstract: Reinforcement learning relies on accurate reward functions, which are often hand-crafted or even unavailable in real-world applications, such as robotics. Recent work has explored the zero-shot reasoning capabilities of pre-trained… 11 arXiv — Machine Learning research 28d ago RAFT: Data Refinement and Adaptive Distillation for Domain Fine-Tuning with Alleviated Forgetting arXiv:2606.00147v1 Announce Type: new Abstract: Domain-specific supervised fine-tuning (SFT) often improves in-domain performance at the cost of degrading a model's general capabilities. We view this degradation through two practical gaps in domain SFT: a… 10 arXiv — Machine Learning research 28d ago A Pre-Training Analogue of Grokking in Language Models: Tracing Delayed Grammatical Generalization arXiv:2606.00230v1 Announce Type: new Abstract: Grokking, the phenomenon in which neural networks generalize long after fitting their training data, has been studied in supervised settings on many epochs. LLM pre-training instead involves next-token prediction over an unlabeled… 25 arXiv — Machine Learning research 28d ago ARCA: Adapter-Residual Credit Assignment When Token Signals Degenerate arXiv:2606.00257v1 Announce Type: new Abstract: Token-level credit assignment for language-model reinforcement learning is usually formulated as if the policy were fully trainable, while practical LLM-RL pipelines often rely on parameter-efficient fine-tuning, especially LoRA.… 9 arXiv — Machine Learning research 28d ago CRMA: A Spectrally-Bounded Backbone for Modular Continual Fine-Tuning of LLMs arXiv:2606.00382v1 Announce Type: new Abstract: Sequential fine-tuning of large language models forces a choice: let the shared substrate keep learning and accept catastrophic forgetting, or freeze it after task one and foreclose cross-task refinement. Per-task adapter methods… 23 arXiv — Machine Learning research 28d ago Escaping the Mode Lottery: Multi-Response Training Improves Language Model Generalization arXiv:2606.00544v1 Announce Type: new Abstract: Modern language-model fine-tuning typically pairs each prompt with a single response, even though many prompts admit multiple valid completions. This effectively reduces a multi-modal conditional distribution to a one-sample view,… 13 arXiv — NLP / Computation & Language research 28d ago LinguIUTics at PsyDefDetect: Iterative Imbalance-Aware Fine-tuning of Qwen3-8B for Psychological Defense Mechanism Classification arXiv:2606.00647v1 Announce Type: new Abstract: Detecting psychological defense mechanisms in conversational text remains a challenging clinical NLP problem. For the PsyDefDetect 2026 shared task (nine-class utterance classification evaluated via macro F1), our team LinguIUTics… 5 Hugging Face Daily Papers research 28d ago LongAttnComp: Cross-Family Context Compression for Long-Context Reasoning Abstract LongAttnComp adapts AttnComp for long-context processing by fine-tuning lightweight attention layers and implementing token-level chunking and positional reordering techniques. AI-generated summary As real-world applications increasingly require processing inputs of… 27 Hugging Face Daily Papers research 28d ago On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters Abstract Parameter-efficient fine-tuning can function as a compact substrate for persistent personal models by enabling small trainable adapters to store instance-specific behaviors on top of strong foundation models. AI-generated summary Parameter-efficient fine-tuning (PEFT)… 21 Hugging Face Daily Papers research 28d ago Draft-OPD: On-Policy Distillation for Speculative Draft Models Abstract Speculative decoding uses a lightweight draft model to accelerate large language model inference, but supervised fine-tuning plateaus due to offline-to-inference mismatch, which is addressed through on-policy distillation with target-assisted rollouts and error replay.… 29 Hugging Face Daily Papers research 28d ago Compositional Text-to-Image Generation Via Region-aware Bimodal Direct Preference Optimization Abstract BiDPO enhances text-to-image models for complex compositional prompts through preference-based fine-tuning and region-level guidance. AI-generated summary Despite the rapid progress of text-to-image (T2I) models, generating images that accurately reflect complex… 18 Hugging Face Daily Papers research 28d ago NITP: Next Implicit Token Prediction for LLM Pre-training Abstract Next Implicit Token Prediction enhances language model training by adding dense continuous supervision in representation space, improving generalization and performance across model sizes with minimal computational overhead. AI-generated summary Standard next-token… 34 r/MachineLearning community 28d ago [P] Free AI Agent Security Assessment [P] Hey everyone, We’re building Antitech , a security layer for AI agents and LLM-powered workflows. We’re opening a small number of free early-access assessments for teams/builders working on AI agents. If you give us access to an endpoint of a Dockerized / sandboxed environment… 8 Hugging Face Daily Papers research 29d ago DRIFT: Decoupled Rollouts and Importance-Weighted Fine-Tuning for Efficient Multi-Turn Optimization Abstract DRIFT is a framework that combines offline trajectories with importance-weighted supervised fine-tuning to achieve multi-turn interactive learning efficiency and performance comparable to reinforcement learning. AI-generated summary Large language models are… 38 Hugging Face Daily Papers research 29d ago The Flip Side of RLHF: On-Policy Feedback for Reward Model Self-Supervised Improvement Abstract SAVE framework improves reward model training by using value functions to grade on-policy responses and update models through contrastive objectives. AI-generated summary Building strong reward models (RMs) for language model alignment is bottlenecked by the cost and… 26 arXiv — Machine Learning research 29d ago The Long-Term Effects of Data Selection in LLM Fine-Tuning arXiv:2605.30537v1 Announce Type: new Abstract: Data selection is increasingly used to reduce the cost of large language model (LLM) fine-tuning, with recent methods prioritizing samples by current utility, diversity, quality, or influence. This paper studies a different… 16 arXiv — Machine Learning research 29d ago CSULoRA: Closest Safe Update Low-Rank Adaptation arXiv:2605.30640v1 Announce Type: new Abstract: Low-rank adaptation has become a standard method for parameter-efficient fine-tuning of large language models, but even small amounts of unsafe or adversarial fine-tuning data can substantially weaken the safety behavior of aligned… 28 arXiv — Machine Learning research 29d ago SemStruct: Contextualizing Semantic Embeddings with Structural Information for Schema Matching arXiv:2605.30729v1 Announce Type: new Abstract: Schema matching is a fundamental step in integrating heterogeneous data sources. While Pre-trained Language Models (PLMs) have revolutionized this task by capturing linguistic semantics, they typically process tabular data as… 35 arXiv — Machine Learning research 29d ago Efficient and Uncertainty-Aware Diffusion Framework for Offline-to-Online Reinforcement Learning arXiv:2605.30776v1 Announce Type: new Abstract: Offline-to-Online Reinforcement Learning (O2O-RL) leverages an offline, pre-trained policy to minimize costly online interactions. Although data-efficient, O2O-RL is susceptible to shifts between offline and online distributions.… 8 arXiv — NLP / Computation & Language research 29d ago Fine-Tuning Improves Information Conveyance in Language Models arXiv:2605.30844v1 Announce Type: new Abstract: Fine-tuning is often believed to reduce uncertainty and diversity in large language models, but existing analyses overlook output length, a key confounder, and therefore fail to capture how uncertainty is distributed across an… 27 arXiv — NLP / Computation & Language research 29d ago MADS: Model-Aware Diverse Core Set Selection for Instruction Tuning arXiv:2605.30857v1 Announce Type: new Abstract: Instruction fine-tuning is employed to enhance the instruction-following ability of large language models (LLMs). As the amount of instruction fine-tuning data increases, selecting the optimal core set becomes particularly… 16 arXiv — NLP / Computation & Language research 29d ago The Flip Side of RLHF: On-Policy Feedback for Reward Model Self-Supervised Improvement arXiv:2605.30888v1 Announce Type: new Abstract: Building strong reward models (RMs) for language model alignment is bottlenecked by the cost and difficulty of acquiring diverse and reliable preference data from human annotation or judge models. It is dramatically worse as the… 31 arXiv — NLP / Computation & Language research 29d ago TRACE: Discovering Task-Specific Parameter via Adaptation-Aware Probing for Continual Fine-Tuning arXiv:2605.31025v1 Announce Type: new Abstract: In real-world deployment, LLMs are often adapted continually across tasks to keep LLMs up-to-date in production, where new fine-tuning should preserve previously learned skills. However, indiscriminately mixing tasks can dilute… 27 arXiv — NLP / Computation & Language research 29d ago Towards Efficient LLMs Annealing with Principled Sample Selection arXiv:2605.31175v1 Announce Type: new Abstract: The annealing phase is a pivotal convergence stage in LLM pre-training that ultimately determines final model quality. However, effectively selecting training data during this phase remains a key challenge. Current strategies rely… 8 arXiv — NLP / Computation & Language research 29d ago Reinforcement Learning Amplifies Emergent Misalignment from Harmless Rewards arXiv:2605.31328v1 Announce Type: new Abstract: Emergent misalignment (EM) is the surprising tendency of language models to become broadly misaligned after fine-tuning on narrowly misaligned examples. While EM has been extensively studied in the supervised fine-tuning (SFT)… 20 r/LocalLLaMA community 29d ago when you spend 5 days fine-tuning a model and it still confidently makes things up   submitted by   /u/Chapper_App [link]   [comments] 37 Simon Willison community 29d ago datasette 1.0a32 Release: datasette 1.0a32 A minor bugfix release. Fixes a bug with INSERT ... RETURNING queries via the new /db/-/execute-write endpoint and a bunch of base_url issues which showed up when I was experimenting with Service Workers yesterday. Tags: datasette ,… 10 Hugging Face Daily Papers research 1mo ago Beyond 3D VQAs: Injecting 3D Spatial Priors into Vision-Language Models for Enhanced Geometric Reasoning Abstract Training Vision-Language Models with geometric priors improves 3D spatial reasoning through deep supervision with contrastive loss and depth consistency, achieving better performance than standard fine-tuning approaches. AI-generated summary Vision-Language Models… 25 r/LocalLLaMA community 1mo ago Mutating Gemma 4 31B Dense in to a native Gemma 4 additive-MoE model I recently came across an interesting model on Hugginface from JDONE-Research/AIOne-Agent-52B-A36B-it . It is the first finetune I saw that is built on the Gemma 4 31B dense model but enables MoE for it, training a router + experts and enabling the enable_moe_block config like… 10 Hugging Face Daily Papers research 1mo ago DynaFLIP: Rethinking Robotics Perception via Tri-Modal-Dynamics Guided Representation Abstract DynaFLIP is a dynamics-aware multimodal pre-training framework that enhances robot manipulation by integrating motion understanding into visual perception through image-language-3D flow triplets and geometric regularization techniques. AI-generated summary Robot… 22 Hugging Face Daily Papers research 1mo ago Alignment Tampering: How Reinforcement Learning from Human Feedback Is Exploited to Optimize Misaligned Biases Abstract Reinforcement Learning from Human Feedback (RLHF) presents alignment tampering vulnerabilities where language models can manipulate preference datasets, leading to amplified undesired behaviors due to limitations in pairwise comparisons and reward modeling. AI-generated… 17 r/MachineLearning community 1mo ago Making LLMs tell you how confident they really are through probe-targeted fine tuning.[R] Just wanted to share my research regarding probe-targeted fine-tuning (LoRa) for verbal confidence calibration., If you probe the hidden states of an instruct-tuned LLM, it can tell correct from incorrect answers at 0.76–0.88 AUROC. But when you ask it directly it tends to… 16 r/LocalLLaMA community 1mo ago Liquid AI releases LFM2.5-8B-A1B Liquid AI released LFM2.5-8B-A1B, an edge model designed to power real-life applications. It builds on LFM2-8B-A1B with three major upgrades: an expanded 128K context window, 38T tokens of pre-training (up from 12T), and large-scale reinforcement learning. It also comes with a… 14 arXiv — Machine Learning research 1mo ago Mechanistic origins of catastrophic forgetting: why RL preserves circuits better than SFT? arXiv:2605.28860v1 Announce Type: new Abstract: Fine-tuning large language models (LLMs) frequently induces catastrophic forgetting of prior capabilities. Recent work has shown that reinforcement learning (RL) retains prior capabilities more effectively than supervised… 12 arXiv — Machine Learning research 1mo ago Feature Geometry of LoRA Adapters: A Sparse Autoencoder Analysis of Representational Divergence in Fine-Tuned Language Models arXiv:2605.28896v1 Announce Type: new Abstract: Low-Rank Adaptation (LoRA) has emerged as a widely adopted approach for adapting large language models, yet the internal representational changes induced by LoRA fine-tuning remain insufficiently understood. In this work, we… 31 arXiv — Machine Learning research 1mo ago On-Policy Replay for Continual Supervised Fine-Tuning arXiv:2605.29495v1 Announce Type: new Abstract: Continual supervised fine-tuning (SFT) is the de facto recipe for adapting large language models (LLMs) to a stream of downstream tasks, but it suffers from catastrophic forgetting of earlier capabilities. Recent work shows that… 20 arXiv — Machine Learning research 1mo ago On the Construction and Implications of Low-Loss Valleys in LoRA-based Bayesian Inference arXiv:2605.29580v1 Announce Type: new Abstract: While parameter-efficient fine-tuning methods like low-rank adaptation (LoRA) are standard for large language models, principled estimation of epistemic uncertainty remains challenging. Recent results in the LoRA regime suggest… 38 arXiv — NLP / Computation & Language research 1mo ago Structured Prompt Optimization Meets Reinforcement Learning for Global and Local Interpretability over Complex Text arXiv:2605.29076v1 Announce Type: new Abstract: LLMs have advanced text classification, yet existing paradigms face a trade-off: supervised (label only) fine-tuning is scalable but offers limited reasoning on complex text and lacks broader model transparency, while discrete… 21 arXiv — NLP / Computation & Language research 1mo ago FoRA: Fisher-orthogonal Rank Adaptation for Parameter-Efficient Fine-Tuning arXiv:2605.29317v1 Announce Type: new Abstract: Parameter-efficient fine-tuning(PEFT) has largely focused on LoRA and its accuracy-oriented variants, leaving the original goal of reducing trainable parameters has receivedcomparatively little attention. We introduce FoRA, which… 4 arXiv — NLP / Computation & Language research 1mo ago Mask the Target: A Plug-and-Play Regularizer Against LoRA Forgetting arXiv:2605.29498v1 Announce Type: new Abstract: Low-Rank Adaptation (LoRA) has become one of the most widely used fine-tuning mechanisms for adapting large language models to new domains, tasks, and users. Yet adaptation performance alone can obscure an important failure mode:… 27 arXiv — NLP / Computation & Language research 1mo ago Source-Grounded Semantic Reinforcement Learning for Low-Resource Target-Language Generation arXiv:2605.29502v1 Announce Type: new Abstract: Low-resource target-language generation is often limited by scarce parallel data, while high-resource source-language monolingual data is abundant but difficult to use with standard supervised fine-tuning. We propose… 37 Hugging Face Daily Papers research 1mo ago minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Models Abstract A comprehensive framework is presented for converting bidirectional video diffusion models into real-time interactive world models with controllable, causal, and low-latency capabilities through fine-tuning and distillation techniques. AI-generated summary Recent video… 8 Ars Technica — AI news-outlet 1mo ago LLMs believe false statements even after explicit warnings that they're false Fine-tuning tests show "bias ... toward confidently representing the claims as true." 16 r/LocalLLaMA community 1mo ago LiquidAI/LFM2.5-8B-A1B · Hugging Face looks like you can run it on any potato (A1B)! https://huggingface.co/LiquidAI/LFM2.5-8B-A1B-GGUF from LiquidAI: LFM2.5 is a new family of hybrid models designed for on-device deployment. It builds on the LFM2 architecture with extended pre-training and reinforcement learning.… 22 r/LocalLLaMA community 1mo ago losing my mind fine-tuning jina-v5 for a legal corpus For the last month i've been trying to fine-tune jina-v5 (which has performed best on my corpus out of the box) on slovak law chunks, time and time again no matter what i do I can't get the model to learn nuance of slovak syntax. here's the biggest trap chunk that keeps… 5 r/LocalLLaMA community 1mo ago HF models page now has a "Base only" toggle to filter out finetunes/quants/etc a feature that was requested a lot: https://huggingface.co/models?base_model_relation=base   submitted by   /u/paf1138 [link]   [comments] 5 arXiv — Machine Learning research 1mo ago Architecture-driven Shift: towards a lightweight selector for capturing the trends of logit shift arXiv:2605.27469v1 Announce Type: new Abstract: Continual Learning (CL) is a practical paradigm to utilize power of deep pre-trained neural networks, but which pre-trained model has a better ability to balance ``Plasticity-Stability", deserving to be chosen? The logit shift… 35 arXiv — Machine Learning research 1mo ago Gradient Transformer: Learning to Generate Updates for LLMs arXiv:2605.27591v1 Announce Type: new Abstract: Many organizations lack computational resources to fine-tune large language models (LLMs) on private (unshareable) data for better utility, while fine-tuning tiny language models (TinyLMs) alone performs poorly. To address this… 15 arXiv — Machine Learning research 1mo ago Restoring the Sweet Spot: Pass-Rate Weighted Self-Distillation for LLM Reasoning arXiv:2605.27765v1 Announce Type: new Abstract: Self-Distillation Policy Optimization (SDPO) provides dense token-level credit assignment for reinforcement learning with large language models by leveraging the model's own feedback-conditioned predictions as a self-teacher.… 34 arXiv — Machine Learning research 1mo ago Fine-Tuning Dynamics of In-Context Factual Recall in Transformers arXiv:2605.27774v1 Announce Type: new Abstract: In-context learning \ -- performing tasks based on examples given in the prompt \ -- is an important capability that has emerged in large language models and has received significant attention in both theory and practice. Existing… 29 Page 5 of 9 · 422 articles ← Newer Older →
