News / #edge Tag Edge 373 articles archived under #edge · RSS Sign in to follow Hugging Face Daily Papers research 1d ago Ready Cohorts: Bounding GPU Opportunity and Avoiding Host Round Trips in LLM-Agent Control Abstract Two studies define measurable GPU control gates for LLM-agent services by analyzing concurrent cohort scheduling and on-device routing versus host redispatch. Generated by thinkingmachines/Inkling-Small LLM-agent services repeatedly execute small deterministic… 25 r/LocalLLaMA community 1d ago LiquidAI/LFM2.5-VL-3B · Hugging Face LFM2.5-VL-3B is a multimodal variant of LFM2.5, a family of hybrid models designed for on-device deployment . It builds on LFM2-VL-3B with further mid- and post-training. LFM2.5-VL-3B can process both text and images, and uses the LFM2.5-2.6B language model as its backbone,… 15 r/LocalLLaMA community 2d ago RAG for regular users? One of the reasons I got into local LLMs was the possibility of getting answers using my own documents and books (a few hundreds) instead of having to search through them manually. However since I'm not a data specialist or an engineer, RAG projects were too difficult for me,… 17 r/LocalLLaMA community 2d ago Anthropic, OpenAI, Google, Meta, Microsoft, and Mistral all signed the EU Code of Practice on Transparency of AI-Generated Content Even open source local models from these companies will be watermarking code and text since it's required by law.   submitted by   /u/Bestlife73 [link]   [comments] 19 r/LocalLLaMA community 3d ago What can us 8 GB VRAM poors do? I want to hook up a local model to Cline, but it seems the best model is still just Qwen 3.5 9B. Please can we have a Qwen 3.8 9B that gets close to Qwen 3.6 27B?   submitted by   /u/Aggravating-Push-207 [link]   [comments] 31 Hugging Face Daily Papers research 3d ago Ego-OSCAR: Egocentric Open source Stereo CAptuRe System Abstract We present Ego-OSCAR, an open-hardware, low-cost, head-mounted stereo-inertial capture device for egocentric data collection in the wild. EgoOSCAR pairs a hardware-synchronized global-shutter stereo camera with a 6- axis IMU, an embedded Linux SBC for on-device video… 28 r/LocalLLaMA community 3d ago I made a web-design benchmark for local models (Muse Glimmer 30B vs Qwen 3.6 27b vs Deepseek V4 Flash 0731)   submitted by   /u/ShadyShroomz [link]   [comments] 34 r/LocalLLaMA community 3d ago Please Share Your Experience About Muse Glimmer I have a classic test for local LLM's. I asked for 8 ball pool game with only one HTML file and Muse Glimmer spend 21k Token(I m using full context so 128k) and only created a 220 lines of HTML and said its done. With my experience its not even close to Qwen 3.6 27B and we are… 19 r/LocalLLaMA community 3d ago Best open-source harness like Claude Code? Avid claude code user here looking to do equivalent things with local models. Just want to plug in something like Qwen and have the interface be 1:1 with claude code. Any suggestion?   submitted by   /u/Neighbor_ [link]   [comments] 36 r/LocalLLaMA community 3d ago Best Local LLMs - August 2026 Wowee!! Just when you thought it couldn't get better for open weight models, we probably have had our best period yet!?!?! Models that rival the closed frontier, Opus level models on non-insane hardware and more. A massive industry alliance coming out in support of open AI in… 25 r/LocalLLaMA community 3d ago Meta open sources new on-device model Muse Glimmer & Muse spark 1.2 also coming soon!   submitted by   /u/provoloner09 [link]   [comments] 17 r/LocalLLaMA community 3d ago I've added Maple-Preview to Mference, got 40 tps generation with 500MB of used RAM on Air M4 I like the idea of running local models, but I don’t like the idea of having them eat up all of my memory. I’ve always thought that the best way to build an edge model would be to make something smart enough to reason over data, but without requiring much knowledge of its own.… 14 arXiv — Machine Learning research 4d ago MAUPITI: On-Device Prototype-Based Learning on a Smart Infrared Sensor arXiv:2608.07192v1 Announce Type: new Abstract: Low-resolution infrared (IR) array sensors represent an interesting solution for privacy-preserving human sensing in embedded systems. In this letter, we describe a smart multi-pixel IR sensor integrating a 16$\times$16 thermal… 13 r/LocalLLaMA community 4d ago small model for text-to-regex Anyone have a good local model for this? i feel like this should be a near solved problem.   submitted by   /u/minaminotenmangu [link]   [comments] 23 r/LocalLLaMA community 5d ago Building a budget 32GB → 48GB VRAM home AI server: 2-3x RX 9060 XT 16GB vs RTX 5060 Ti 16GB, AM5 vs used EPYC? I’m planning a dedicated home AI server, mainly for local LLM inference, agents/tool use, Docker services, and eventually larger MoE models with CPU offload. My plan is to start with 2x 16GB GPUs = 32GB VRAM , but I want to build the platform from day one knowing that I’ll… 37 r/MachineLearning community 6d ago Built a tool to generate slides from research papers using local LLMs (because I hate formatting decks and privacy matters) [P] Hi guys, Every time I had to prepare a presentation based on a paper or research doc, I found the process super tedious. Plus, I really dislike uploading unpublished stuff or sensitive data to online AI services just to get a draft. So I put together a tool called… 22 r/LocalLLaMA community 7d ago what will be the future of LocalLLaMA? For a long time now, the most popular posts on LocalLLaMA have been either about using LLM in the cloud or about politics. I suspect that people using local models are about 10% now. You can say that this is very good, because now it is an inclusive sub, without gatekeeping. But… 28 arXiv — NLP / Computation & Language research 7d ago EdgeXpert: An Edge Device for Memory-Efficient LLM Inference with Mixture-of-Experts and Speculative Decoding arXiv:2608.05303v1 Announce Type: cross Abstract: On-device deployment of Large Language Models (LLMs) has become essential for personalized edge applications. A primary bottleneck is external memory access (EMA) in feed-forward network (FFN) layers. Speculative decoding and… 21 r/LocalLLaMA community 7d ago 🟩 NVIDIA's whole speech stack just went local. ASR + TTS + codec, quantized to GGUF, running on-device via NeMo-Speech.cpp 🐦⬛ Magpie-TTS Multilingual 🦜 Nemotron Speech Streaming EN 0.6B 🦜 Nemotron-3.5 ASR Streaming 🦜 Parakeet CTC 1.1B 🦜 Parakeet TDT 0.6B v3 🥦 NanoCodec Merged PR https://huggingface.co/nvidia/magpie_tts_multilingual_357m#run-magpietts-locally-with-nemo-speechcpp I run open… 9 r/LocalLLaMA community 7d ago I thought Deepseek was the answer since I cannot afford GPU for local LLM   submitted by   /u/HsSekhon [link]   [comments] 18 r/LocalLLaMA community 7d ago Best open-source harnesses for combining cloud and local AI model orchestration? Looking for best current solutions for combining cloud models and local models seamlessly inside a harness' orchestration Edit: Right now, we don't have harnesses (that I'm aware of) that are blending local and cloud models to work together simultaneously to accomplish tasks set… 32 r/LocalLLaMA community 7d ago 32 total local models tested head to head I ran 32 local models head to head on one fact-extraction corpus, 1,001 notes, paired bootstrap on every adjacent pair. Several weeks of compute time, all on consumer grade cards. Most of the field does not separate. Six consecutive steps from 2B to 31B, and the bootstrap cannot… 37 r/LocalLLaMA community 8d ago i just spent weeks rewriting my webUI from scratch, getting rid of all AI slop within the codebase and switching it over to a proper lightweight framework (alpine.js). i am now comfortable suggesting it as an alternative to openwebUI, librechat and the like! it is made for local… [Fully open source under GPL3, made from the ground up for use with local models, no subscriptions, no corporate backing] When i first started this, it was meant to be a fully lightweight, extremely modular alternative to openclaw, hermes and the like , and it still is! But i… 19 arXiv — NLP / Computation & Language research 8d ago Energy- and Memory-Efficient PEFT Methods for Personalized On-Device SLMs on Consumer GPUs arXiv:2608.04488v1 Announce Type: new Abstract: Despite rapid advances in large language models (LLMs), deploying and personalizing them on resource-constrained devices remains impractical due to high VRAM, time, and energy costs. Parameter-Efficient Fine-Tuning (PEFT) of Small… 8 r/MachineLearning community 8d ago Running Whisper, Qwen3-ASR, Nemotron & MOSS completely offline on iPhone [P] Over the past month, I've been building LiveTranscriber, an open-source iOS app for running modern speech and language models entirely on-device. The goal was to see whether recent open-source models could be turned into a practical mobile product—not just technical demos.… 8 r/LocalLLaMA community 8d ago Ling-3.0-flash MXFP4 released and running locally on one DGX Spark. In tests: ~80 tok/s decoding 2,500–3,500 tok/s long-input prefilling Smooth use by 3–4 concurrent users Private, on-device inference for coding, agents, and offline batch jobs   submitted by   /u/niacolhealth [link]   [comments] 30 r/LocalLLaMA community 8d ago Given the MiniMax H3 LoRAs Debacle - Some Important Context for Censorship enforcement and laws in China *I felt the need to write this post because it seems like very few people on this sub are aware of Chinese laws and how they're enforced, so here's an explainer coming from a Chinese person (myself). I know that this post isn't directly about local models per se, but I'm seeing… 12 TechCrunch — AI news-outlet 8d ago MacPaw taps Liquid AI to offer on-device inference to devs building for its app store MacPaw is building a local version of its AI assistant Eney using Liquid AI's models. 33 r/LocalLLaMA community 9d ago Building a Fully Local PDF Read-Aloud & PDF-to-Audiobook Desktop App with Kokoro 82M, Qwen, and llama.cpp Hey everyone, I’ve been building Speechfony - a desktop app for reading PDFs (and EPUBs) with offline text-to-speech. Open a document, listen sentence-by-sentence with highlighting, or export selected pages to an MP3. Everything runs locally: Kokoro for speech, and an on-device… 5 arXiv — Machine Learning research 9d ago GLOBE: Trajectory-Aligned Gradient Matching with Structured SparseOptimization for Coreset Selection arXiv:2608.02690v1 Announce Type: new Abstract: On-device training of deep neural networks is fundamentally constrained by the computational and memory costs of large-scale datasets. Coreset selection offers a practical solution by retaining only a compact subset of real… 22 arXiv — Machine Learning research 9d ago Design-Time Optimization of Deep Neural Networks for Intermittent Learning on Microcontrollers arXiv:2608.03589v1 Announce Type: new Abstract: We present a method for designing deep neural networks (DNNs) for intermittent, energy-autonomous, on-device learning on microcontroller units (MCUs). In mobile applications where the energy can run out, e.g., when solar-powered,… 10 arXiv — NLP / Computation & Language research 9d ago MemArena: An Ego-Centric Benchmark for On-Device Agentic Personal Memory Assistants at Scale arXiv:2608.02613v1 Announce Type: new Abstract: Edge-deployed personal memory assistants must handle private interpersonal conversations on-device with open-weight models. Yet, existing memory benchmarks often under-test the combination of activity-dense interaction, ego-centric… 15 r/LocalLLaMA community 9d ago GPT-OSS has turned one year old today! It is one of the best local models ever released, in both 20B and 120B versions. I always come back to it, especially the 120B version. Its only competition is, in my opinion, Qwen 3.5 122B, but that model is much slower (A10B) and has not been released in a local-friendly QAT… 9 r/LocalLLaMA community 9d ago Local LLM 35B MoE — Real-world coding benchmarks (Qwen vs Ornith vs KAT) I’ve been running a fairly opinionated evaluation loop on ~35B A3B/MoE-class models for coding over the past few months. Not synthetic benchmarks: actual dev workflows, iterative debugging, refactoring passes, and failure recovery. Here’s where things stand for me: Qwen 3.6 (35B… 9 r/LocalLLaMA community 9d ago SK hynix, In Collaboration With SanDisk, Unveils The New High Bandwidth Flash (HBF) Standard, Helping To Resolve AI Inference Bottlenecks, Targeting Up To 3TB/s Bandwidth Hopefully this would let us have faster local models....but it will probably be out of our price range.   submitted by   /u/giveen [link]   [comments] 12 r/LocalLLaMA community 10d ago Is LM Studio abandoning their core product? Some of you may be aware that a few weeks ago, LM Studio announced a new agent, Bionic. This is pretty much an agentic harness for both local models and paid cloud models. But most aren't aware that LM Studio replaced almost every link to the original app that built their brand… 7 llama.cpp releases dev-tools 10d ago b10255 Extended SYCL oneDNN SDPA to non-FP16 KV caches (Q4_0–Q8_0 and FP32) ( #25874 ) sycl: extend oneDNN SDPA to Q4_0-Q8_0 and F32 KV caches Extends the oneDNN SDPA path (PR #25222 ) to handle non-F16 KV caches by dequantizing or converting K/V to dense FP16 on-device before feeding… 4 arXiv — Machine Learning research 10d ago Kilobyte Models: Neural Networks as a Seed and a Quantized Latent arXiv:2608.00860v1 Announce Type: new Abstract: The cost of storing and transmitting a trained neural network scales with its parameter count, a bottleneck for over-the-air updates, on-device libraries, and other bandwidth-bound deployments. We study an extreme form of model… 17 arXiv — NLP / Computation & Language research 10d ago Opt.Gear Technical Report arXiv:2608.01034v1 Announce Type: new Abstract: We introduce Opt.Gear, a foundation model designed for efficient on-device deployment, real-tim inference, and strong task capability. It includes a dense model (1M, 270M, and 1B) with a context length of 64K. We designed a new… 32 arXiv — Machine Learning research 11d ago Versatile On-device Adaptation at the Edge by Unifying Few-shot, Zero-shot, Continual, and In-context Learning arXiv:2607.29353v1 Announce Type: new Abstract: With the ever-increasing pervasiveness of smart edge devices, the demand is growing for applications that can be tailored to users (e.g., custom keyword spotting) or patients (e.g., adaptive health monitoring). Yet, most edge… 38 arXiv — Machine Learning research 11d ago GQ-FSL: Green Quantized Federated Split Learning arXiv:2607.29659v1 Announce Type: new Abstract: Deploying state-of-the-art deep neural networks (DNNs) at the wireless edge is severely bottlenecked by the strict energy and resource constraints of mobile devices. While federated split learning (FSL) mitigates on-device… 28 arXiv — NLP / Computation & Language research 11d ago Data Turnstile: A Scalable Open Framework for Function-Calling Data Generation arXiv:2607.29250v1 Announce Type: new Abstract: Small language models (SLMs) are attractive for agentic deployment due to low latency, reduced cost, and on-device privacy, yet they struggle with tool-use tasks where training data is scarce and noisy. Unlike larger models, SLMs… 13 r/LocalLLaMA community 12d ago A collection of small domain-specific benchmarks for local models (30+ and growing) Hello fellow local AI people! I took "you must create your own benchmarks" literally, and built a website for this. How does the end result look like Let's say I want to know which model has most common sense in its responses, I did everything including evaluating responses (see… 34 r/LocalLLaMA community 12d ago Tomte - super fast harness for Gemma 4 I think people are sleeping on Gemma and local models so I built a free, very fast harness for Gemma 4 that I call Tomte. https://tomteapp.com Works on Macs with M processors, will have a companion app you can connect to anywhere. So far does everything I ever needed chatGPT… 5 r/LocalLLaMA community 12d ago I'm kinda tired of obsession for one-shot tests in coding, there are good tests for multi-step debugging with analyzing output/images/videos? Personally, i think good coding model shouldn't be focused on one-shot "everything in one html-file" tests, but should be really good on debugging, fixing and modifying its own output. Anyone know such simple tests that i would able to run with local models? May be some kind of… 15 r/LocalLLaMA community 13d ago Rule Suggestion: "Open" models without weight releases should be tagged [no weights] A lot of recent models are being announced with promised open weights, but the weights are either weeks away, or in some cases (looking at you Meta) not being released at all. This sub is about local LLMs - not "maybe local in the future" llms. These models are still useful to… 26 Hugging Face Daily Papers research 14d ago AMRD: Adaptive Multi-Teacher Relational Distillation for Lightweight Speech Emotion Recognition Abstract On-device speech emotion recognition (SER) is critical for real-time applications, yet large self-supervised models that excel at SER are too costly for edge devices. Multi-teacher knowledge distillation can compress them into a lightweight student, but two challenges… 17 arXiv — Machine Learning research 14d ago DAS-PMVC: A Framework for Partial Multi-View Clustering via Dual Alignment and Structure Enhancement arXiv:2607.27761v1 Announce Type: new Abstract: In recent years, multi-view clustering has attracted widespread research interest. However, due to limitations in data collection devices, data across different views often suffer from misalignment, leading to the partial view… 14 arXiv — NLP / Computation & Language research 14d ago Gradient-free Task-Conditioned Retrieval for On-Device In-Context Learning arXiv:2607.27766v1 Announce Type: new Abstract: On-device in-context learning (ICL) relies on pre-inference retrieval to select demonstrations for useful context before downstream model inference. This retrieval must exploit task-specific information while operating over local… 21 r/LocalLLaMA community 14d ago Local LLMs for non-coding What are your top 3 uses cases? Seems that outside coding the application of local is limited?   submitted by   /u/Salt_Armadillo8884 [link]   [comments] 18 Page 1 of 8 · 373 articles Older →
