r/LocalLLaMA

500 articles archived · Visit source ↗ · RSS

r/LocalLLaMA community 3d ago

Qwen 3.8-27b coming this week

Confirmed by the official Qwen account.   submitted by   /u/Bestlife73 [link]   [comments]

25
r/LocalLLaMA community 3d ago

Nvidia reportedly testing lower memory configs of Rubin Ultra as memory shortage bites back — designs tested include as little as 192 GB and step back to HBM4

  submitted by   /u/fallingdowndizzyvr [link]   [comments]

16
r/LocalLLaMA community 3d ago

I gave DeepSeek V4 Flash basic vision by training a 40M connector on 100K examples

I wanted to find out whether a huge text-only MoE could be given basic vision without retraining the language model itself. The short answer is yes. I froze DeepSeek V4 Flash and a 417M-parameter MoonViT image encoder, then trained a 40.1M-parameter connector between them on…

16
r/LocalLLaMA community 3d ago

1 Day in and I feel okay saying Muse-Glimmer-30B finally beats 3.6-27B for the size in some use-cases

A few things right off the bat: it reasons very efficiently. Like Grok 4.5 levels of efficient thinking it quantizes very well. My first few tests with iq3_xxs were better than Qwen/Gemma behaved at that size its knowledge depth is amazing. It beats Qwen3.6 27B on no-tools…

14
r/LocalLLaMA community 3d ago

Muse-Glimmer 30B Hits ~280 t/s in Real Production Coding

These numbers were captured during a real feature implementation task in Next.js and Nest.js (adding a theme switching system across components). The structural predictability of UI/state refactoring is likely why DFlash hit such a high draft acceptance rate (~97%). Here is a…

6
r/LocalLLaMA community 3d ago

Observations on Muse-Glimmer reasoning traces being noticeably different from qwen / gemma models and questions for you guys

Just downloaded the model, UD-Q5_K_XL quant, asked it to generate a long story to test out reasoning and speed with dflash (super fast btw, ~ 90 to 160 tok/s on a 5090 depending on task) and was surprised at the reasoning traces, they are so unlike anything i've seen recently…

5
r/LocalLLaMA community 3d ago

I trained a 1B-parameter LLM from scratch on 20B tokens for about $200

A few months ago, I had the idea of making a LLM from scratch as a personal project (for learning and partly for improving my resume). Since I learned a lot from other posts on here over the past year, I wanted to share the results. TLDR: I trained a 1.1B param model on 20B…

11
r/LocalLLaMA community 3d ago

Muse glimmer benchmark

Little less smart than Qwen, but way fewer tokens per task.   submitted by   /u/NoFaithlessness951 [link]   [comments]

18
r/LocalLLaMA community 3d ago

Muse Spark 1.2 Open Source before Llama 4 Behemoth!!?

I can’t believe it!! When Muse Spark just came out, I was already thinking they might consider open sourcing this. And now they’re actually gonna open source it!! And ever since Alexandr Wang took over, they’d be releasing anything but Llama 4 Behemoth! What’s next? Llama 5…

38
r/LocalLLaMA community 3d ago

I made a web-design benchmark for local models (Muse Glimmer 30B vs Qwen 3.6 27b vs Deepseek V4 Flash 0731)

  submitted by   /u/ShadyShroomz [link]   [comments]

34
r/LocalLLaMA community 3d ago

Tested Muse Glimmer locally on coding with OpenCode & agentic work

Ran the model with quants (Q4) by Unsloth with latest (build from master) llama.cpp server. It takes ~20GB ram running on M5 Pro with 48GB at about 17t/s. Didn't do any reasoning loops/overthinking. Overall, sits below Qwen3.6 27B, wasn't able to get good code (frontend and…

30
r/LocalLLaMA community 3d ago

Achievable 253 t/s - unsloth/Muse Glimmer 30B UD-Q5_K_M on a 5090

Benchmarked Muse Glimmer 30B on my RTX 5090 (32GB), 262k context, UD-Q5_K_M + dflash-kquant + mmproj. Workload Stock master + DFlash ngram-simple PR #26842 + DFlash Code patch 78 t/s 57 t/s 220-253 t/s Mixed agent turn 77 t/s 68 t/s 188-213 t/s Tool-call JSON 71 t/s 75 t/s…

15
r/LocalLLaMA community 3d ago

Please Share Your Experience About Muse Glimmer

I have a classic test for local LLM's. I asked for 8 ball pool game with only one HTML file and Muse Glimmer spend 21k Token(I m using full context so 128k) and only created a 220 lines of HTML and said its done. With my experience its not even close to Qwen 3.6 27B and we are…

19
r/LocalLLaMA community 3d ago

Best open-source harness like Claude Code?

Avid claude code user here looking to do equivalent things with local models. Just want to plug in something like Qwen and have the interface be 1:1 with claude code. Any suggestion?   submitted by   /u/Neighbor_ [link]   [comments]

36
r/LocalLLaMA community 3d ago

I compared GGUF quants of Qwen3.6 27B to NVFP4, AWQ, AutoRound, and FP8

There's an interactive chart and some extra data in the blog post if you're interested. There are plenty of KL-divergence benchmarks for GGUF models, but most of them compare one GGUF quant against another. I wanted to know how those quants stack up against other commonly used…

28
r/LocalLLaMA community 3d ago

Glimmer: 233.4 tps on 5090 with Dflash

That's insane yo! I haven't had a chance yet to test it on my 4090 at home but it sounds so promising. And read here that 256k CTX is easily reachable on 24gb unlike Qwen. Super excited!   submitted by   /u/YetAnotherAnonymoose [link]   [comments]

24
r/LocalLLaMA community 3d ago

Needle 2: 14MB agentic LLM for phones, wearables, smart home and robots.

Hey LocalLlaMa, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now…

15
r/LocalLLaMA community 3d ago

inclusionAI/Ling-3.0-tiny · 8B A1.3B MoE· Hugging Face

Looks like the Ling team open weighted a much smaller version of the Ling-3.0-flash they open weighted a few days ago. It's 8B params with 1.3B active, and seems to fall between the 4B and 8-12B Qwen and Gemma models in terms of performance. Should have a massive tokens/sec on…

33
r/LocalLLaMA community 3d ago

DiffusionGemma Technical Report

arXiv : https://arxiv.org/abs/2608.00146 Full Paper : https://arxiv.org/pdf/2608.00146 Tweet : https://xcancel.com/googlegemma/status/2086849199052845451#m FYI both (llama.cpp) PRs ( 24423 & 24427 ) went to Draft mode. I'm still waiting for this one as I could get faster t/s on…

15
r/LocalLLaMA community 3d ago

DeepSeek V4 Flash 0731 is the ‘killer app’ that is going to sell A LOT of DGX Sparks

Having a ‘Killer Application’ that everyone wants to use helps sell hardware, plain and simple. DeepSeek V4 Flash 0731 isn’t an app of course, but I think it’s going to be the major catalyst for getting a lot of people to buy a couple of NVIDIA GB10-based systems because: It is…

33
r/LocalLLaMA community 3d ago

Ante 0.2: a ~15MB coding agent that manages llama.cpp for you — point it at a GGUF and the whole agent loop runs offline

Hello~ We just shipped Ante 0.2, and the part I think this community will care about most is offline mode. We wanted local to be a first-class way to run the agent, so Ante manages the inference engine itself: Managed llama.cpp. Ante installs a pinned, checksum-verified official…

7
r/LocalLLaMA community 3d ago

Early signs that Muse-Glimmer-30B might quantize *very* well? Share your experiences.

  submitted by   /u/EmPips [link]   [comments]

4
r/LocalLLaMA community 3d ago

Best Local LLMs - August 2026

Wowee!! Just when you thought it couldn't get better for open weight models, we probably have had our best period yet!?!?! Models that rival the closed frontier, Opus level models on non-insane hardware and more. A massive industry alliance coming out in support of open AI in…

25
r/LocalLLaMA community 3d ago

Muse Glimmer ACTUALLY fits on a single RTX 3090

I did some testing this morning, and I was surprised to find that Muse Glimmer actually comfortably fits on a single RTX 3090 with full context + DFlash + mmproj at Q4_K_XL, unlike Qwen3.6-27B and Gemma-4-31B. Muse Glimmer supports up to 256k context according to Unsloth. Here…

14
r/LocalLLaMA community 3d ago

Motif-Technologies/Motif-3 official realese

Motif-Technologies is one of the tech company participated South Korea's AI Foundation Model project.(독파모) Upstage(Solar Series), LG AI Research(EXAONE Series), and SKT(A.X Series) are the competitors. Since LG’s EXAONE put up pretty disappointing results, it looks like Upstage,…

12
r/LocalLLaMA community 3d ago

Glimmer seems pretty censored?

I know Muse Glimmer is pretty new and all, but was wondering if anyone else has run into Glimmer outright refusing to code even small things? I am using Unsloth Q8, dual 3090's, in Kilo Code. I was trying to get it to help me with a bug in my codebase (using pyton stdlib to…

8
r/LocalLLaMA community 3d ago

Mark Zuckerberg on releases

https://x.com/i/status/2086755195535413696   submitted by   /u/jacek2023 [link]   [comments]

9
r/LocalLLaMA community 3d ago

model: Muse Glimmer Support by pcuenca · Pull Request #26841 · ggml-org/llama.cpp

Day 0 support   submitted by   /u/jacek2023 [link]   [comments]

19
r/LocalLLaMA community 3d ago

Native Long Video Understanding Models locally?

I've been building a personal project and wanted to check with the community on multi-modal inputs since I can't find a lot of material around this online. Ultimately I'm trying to build something that can ingest massive length (almost like a full stream - 6-10 hours) and…

21
r/LocalLLaMA community 3d ago

1M context with 17 GB model in 24 GB VRAM: "for the first time I was able to load a context of almost 1M tokens and extract 7 needles from various parts of the text"

https://preview.redd.it/xxjh11f38jih1.png?width=1852&format=png&auto=webp&s=76850ed51e29a8bc86c2ca718d4320075eed4363 Just wanted to share a user report that I found to be very interesting. Some person with an intriguing name manu69x managed to run 1M context on a single RTX 3090…

30
r/LocalLLaMA community 4d ago

unsloth/Muse-Glimmer-30B-GGUF · Hugging Face

Guide: https://unsloth.ai/docs/models/muse-glimmer   submitted by   /u/Nunki08 [link]   [comments]

28
r/LocalLLaMA community 4d ago

Comparing how Cline, Kilo, and Qwen Code handle long-task context/state (and why context loops keep happening)

I've been comparing Cline / Kilo / Qwen Code lately since they all handle long-task state differently. Cline: has Focus Chain, a markdown file kept outside the conversation that gets reinjected on a cadence, plus Memory Bank for project context, plus a standalone gRPC server so…

31
r/LocalLLaMA community 4d ago

Introducing Muse Glimmer: an open-weight model optimized for always-on local agent workflows

Hi r/LocalLLaMA 👋 Today we’re excited to release Muse Glimmer, a 30B open-weight model built specifically for local agent workflows. We’re releasing the weights to the community under a permissive Apache 2.0 license. A few specs 30B params, dense Multimodal: interleaved text +…

35
r/LocalLLaMA community 4d ago

Meta open sources new on-device model Muse Glimmer & Muse spark 1.2 also coming soon!

  submitted by   /u/provoloner09 [link]   [comments]

17
r/LocalLLaMA community 4d ago

Mature/better alternatives to pi-web?

I'm looking for a web interface for my pi harness without the CLI, but this was the only one I found. Are there any others? I'm not looking for any chat interface like Open WebUI that's rather bloated, requiring accounts/logins for certain features, etc. It would be great if it…

26
r/LocalLLaMA community 4d ago

I've added Maple-Preview to Mference, got 40 tps generation with 500MB of used RAM on Air M4

I like the idea of running local models, but I don’t like the idea of having them eat up all of my memory. I’ve always thought that the best way to build an edge model would be to make something smart enough to reason over data, but without requiring much knowledge of its own.…

14
r/LocalLLaMA community 4d ago

Chat UIs with native audio input for multimodal models?

I've been running Gemma 4 E4B with oMLX and I can't find any chat interfaces that directly send the audio file to the model instead of running the audio through a separate STT layer. I can confirm the audio layers work because I ran a couple of requests through Pydantic AI in…

6
r/LocalLLaMA community 4d ago

Why Speculative Decoding went mature in 2026?

Spec-dec has been a thing for a while, in fact, it's wasn't an idea that was born for LLM inference. E.g. Uber's https://github.com/uber/submitqueue applied it to a merge queue. Apple & GDM had been releasing papers on it since already 2022. Seeing it being mature enough for the…

16
r/LocalLLaMA community 4d ago

MiniMax H3: A New Open-Weight Video Model, Live in ComfyUI

MiniMax H3 is an open-weight, general-purpose multimodal video generation model that works across text, images, video, and audio. In ComfyUI, you can use H3 for text-to-video, image-to-video, first- and last-frame generation, and reference-driven creation. H3 jointly generates…

10
r/LocalLLaMA community 4d ago

Running Qwen 3.5 35B A3B-Q8_0 gguf on a cheap radeon 7600 at 18 token/s

I also have 64 gb ddr4 ryzen 5600 Using llama.cpp Ubuntu distro Settings are as follows --n-gpu-layers 999 \ --n-cpu-moe 37 \ --no-mmap \ -ctk q8_0 \ -ctv q8_0 \ -fa 1 \ -c 9000 \   submitted by   /u/Sweaty_Perception655 [link]   [comments]

30
r/LocalLLaMA community 4d ago

omlab/VLX-Seek-1.5-10B · Hugging Face

VLX-Seek-1.5-10B VLX-Seek-1.5-10B is the open-source 10B model in the VLX-Seek 1.5 family, designed for fine-grained perception and visual grounding in embodied scenarios. It targets practical settings such as drones, robots, robotic dogs, surveillance cameras, inspection…

27
r/LocalLLaMA community 4d ago

Optimal Configuration for 4x3090s

Aiming for RTX 6000 like performance at 25% of the cost. https://preview.redd.it/mi6fqpdd5hih1.png?width=631&format=png&auto=webp&s=9a639bcc79c7eb0a3834a88317220c289b7a52b6 The top 2x3090s are connected via tensor parallelism, then those are connected in a pipeline feeding into…

12
r/LocalLLaMA community 4d ago

So... did we give up on the rule against AI posts?

Sub is drowning in slop posts. Shortly after the new rule it was better. But it's gotten unbearable in the past month or so.   submitted by   /u/kevin_1994 [link]   [comments]

37
r/LocalLLaMA community 4d ago

Need real world ML problems to evaluate my educational ML tools

I'm a retired platform engineer, coding mainly in Rust, and involved with a ML study group. I developed a ML programming language (alternative to Python, Colab) to help me learn (and teach) ML concepts, visually. It has slowly grown to demonstrate a lot of concepts. It can do…

22
r/LocalLLaMA community 4d ago

ByteDance vows to avoid AI distillation, develop new model its own way

  submitted by   /u/etherd0t [link]   [comments]

29
r/LocalLLaMA community 4d ago

KPMG Says Nearly Half Of Executives Pulled Back AI Agents Over Cost

https://www.forbes.com/sites/sandycarter/2026/08/09/kpmg-says-nearly-half-of-executives-pulled-back-ai-agents-over-cost/ Bubble started to burst?   submitted by   /u/MoodDelicious3920 [link]   [comments]

16
r/LocalLLaMA community 4d ago

[NEW MODEL] SupraElegans-500K

*SupraLabs released a new experimental model!\ * SupraElegans-500K is a ~500,000-parameter causal language model built around a sparse, signed, recurrent neural graph. No Transformer, no attention mechanism, no positional encoding, no KV cache. Context is carried by a persistent…

28
r/LocalLLaMA community 4d ago

KLQ: Training-free measured rotation quantization. Beats all training-free rotation-based quantization methods on W4A4KV4-bits. Llama 3.2 1B KLQ-quantized beats SpinQuant and gets close to ReSpinQuant without GPTQ/LDLQ rounding.

First of all, I'm not a lab, this was a solo summer research project that finally culminated into the github repo and the writeup. The repo includes a much deeper dive with methods, findings about quantization and geometry, limitations, and proposed experiments. I'll also…

32
r/LocalLLaMA community 4d ago

endless-frontier/BigBang-v1 - qwen 3.5 finetunes

table bench https://huggingface.co/bartowski/endless-frontier_BigBang-v1-GGUF I'm downloading this model only because Bartowski converted it to .gguf, so it might be interesting. Doubts : The headline number is basically meaningless. "Performance between DeepSeek Flash (old one)…

26
r/LocalLLaMA community 4d ago

The Gemma team will host a special event on August 20

Tweet by u/hackerllama Could be copium, but I would love to see Gemma 4.1 there with unified audio input for all model sizes perhaps even up to 120B, much improved tool calling (even with the latest template there are still bugs ), higher precision QAT from the start and…

20

Qwen 3.8-27b coming this week

Nvidia reportedly testing lower memory configs of Rubin Ultra as memory shortage bites back — designs tested include as little as 192 GB and step back to HBM4

I gave DeepSeek V4 Flash basic vision by training a 40M connector on 100K examples

1 Day in and I feel okay saying Muse-Glimmer-30B finally beats 3.6-27B for the size in some use-cases

Muse-Glimmer 30B Hits ~280 t/s in Real Production Coding

Observations on Muse-Glimmer reasoning traces being noticeably different from qwen / gemma models and questions for you guys

I trained a 1B-parameter LLM from scratch on 20B tokens for about $200

Muse glimmer benchmark

Muse Spark 1.2 Open Source before Llama 4 Behemoth!!?

I made a web-design benchmark for local models (Muse Glimmer 30B vs Qwen 3.6 27b vs Deepseek V4 Flash 0731)

Tested Muse Glimmer locally on coding with OpenCode & agentic work

Achievable 253 t/s - unsloth/Muse Glimmer 30B UD-Q5_K_M on a 5090

Please Share Your Experience About Muse Glimmer

Best open-source harness like Claude Code?

I compared GGUF quants of Qwen3.6 27B to NVFP4, AWQ, AutoRound, and FP8

Glimmer: 233.4 tps on 5090 with Dflash

Needle 2: 14MB agentic LLM for phones, wearables, smart home and robots.

inclusionAI/Ling-3.0-tiny · 8B A1.3B MoE· Hugging Face

DiffusionGemma Technical Report

DeepSeek V4 Flash 0731 is the ‘killer app’ that is going to sell A LOT of DGX Sparks

Ante 0.2: a ~15MB coding agent that manages llama.cpp for you — point it at a GGUF and the whole agent loop runs offline

Early signs that Muse-Glimmer-30B might quantize *very* well? Share your experiences.

Best Local LLMs - August 2026

Muse Glimmer ACTUALLY fits on a single RTX 3090

Motif-Technologies/Motif-3 official realese

Glimmer seems pretty censored?

Mark Zuckerberg on releases

model: Muse Glimmer Support by pcuenca · Pull Request #26841 · ggml-org/llama.cpp

Native Long Video Understanding Models locally?

1M context with 17 GB model in 24 GB VRAM: "for the first time I was able to load a context of almost 1M tokens and extract 7 needles from various parts of the text"

unsloth/Muse-Glimmer-30B-GGUF · Hugging Face

Comparing how Cline, Kilo, and Qwen Code handle long-task context/state (and why context loops keep happening)

Introducing Muse Glimmer: an open-weight model optimized for always-on local agent workflows

Meta open sources new on-device model Muse Glimmer & Muse spark 1.2 also coming soon!

Mature/better alternatives to pi-web?

I've added Maple-Preview to Mference, got 40 tps generation with 500MB of used RAM on Air M4

Chat UIs with native audio input for multimodal models?

Why Speculative Decoding went mature in 2026?

MiniMax H3: A New Open-Weight Video Model, Live in ComfyUI

Running Qwen 3.5 35B A3B-Q8_0 gguf on a cheap radeon 7600 at 18 token/s

omlab/VLX-Seek-1.5-10B · Hugging Face

Optimal Configuration for 4x3090s

So... did we give up on the rule against AI posts?

Need real world ML problems to evaluate my educational ML tools

ByteDance vows to avoid AI distillation, develop new model its own way

KPMG Says Nearly Half Of Executives Pulled Back AI Agents Over Cost

[NEW MODEL] SupraElegans-500K

KLQ: Training-free measured rotation quantization. Beats all training-free rotation-based quantization methods on W4A4KV4-bits. Llama 3.2 1B KLQ-quantized beats SpinQuant and gets close to ReSpinQuant without GPTQ/LDLQ rounding.

endless-frontier/BigBang-v1 - qwen 3.5 finetunes

The Gemma team will host a special event on August 20

Early signs that Muse-Glimmer-30B might quantize very well? Share your experiences.