r/LocalLLaMA

500 articles archived · Visit source ↗ · RSS

r/LocalLLaMA community 1d ago

DeepSeek-V4-Pro-0813 is UP!

it's up now！   submitted by   /u/shing3232 [link]   [comments]

27
r/LocalLLaMA community 1d ago

Qwen3.8-Max

  submitted by   /u/frontsideair [link]   [comments]

11
r/LocalLLaMA community 1d ago

Qwen3.8-2.4T-A95B Released

  submitted by   /u/de4dee [link]   [comments]

17
r/LocalLLaMA community 1d ago

Qwen/Qwen3.8-2.4T-A95B · Released!

  submitted by   /u/CodeCrusader24 [link]   [comments]

12
r/LocalLLaMA community 1d ago

Qwen 3.8 2.4T is out , no 27b today RIP.

i was hoping they would release both today but the big one just dropped now , and it says its been listed since 5 hours ago on huggingface. I guess the real date for 27b is https://modelscope.cn/models/Qwen/Qwen3.8-27B   submitted by   /u/cviperr33 [link]   [comments]

32
r/LocalLLaMA community 1d ago

LiquidAI/LFM2.5-VL-3B · Hugging Face

LFM2.5-VL-3B is a multimodal variant of LFM2.5, a family of hybrid models designed for on-device deployment . It builds on LFM2-VL-3B with further mid- and post-training. LFM2.5-VL-3B can process both text and images, and uses the LFM2.5-2.6B language model as its backbone,…

15
r/LocalLLaMA community 1d ago

Exact Qwen 3.8 27b release date and time

Since it seems like there is some confusion in other threads... Source: https://modelscope.cn/models/Qwen/Qwen3.8-27B   submitted by   /u/yuicebox [link]   [comments]

33
r/LocalLLaMA community 1d ago

NVIDIA's Fastest Blackwell GPU, the 96 GB RTX PRO 6000, Now Costs $16,000, Almost Double Its Original Price

  submitted by   /u/ab2377 [link]   [comments]

6
r/LocalLLaMA community 1d ago

All your reasoning are belong to us

  submitted by   /u/indicava [link]   [comments]

20
r/LocalLLaMA community 1d ago

Hidden Reasoning from Claude and GPT are Decoded, and it is interesting

Yesteday a paper showed a gap that allows to see 100% of the reasoning tokens form ALL Claude and GPT models Stealing Reasoning Traces from Proprietary LLM APIs . check it out, they have published lots of example reasonings. this is very relevant for open soruce; for the…

36
r/LocalLLaMA community 1d ago

Intel LLM-Scaler ready with Muse Glimmer support, other LLMs & features

  submitted by   /u/Fcking_Chuck [link]   [comments]

5
r/LocalLLaMA community 1d ago

According to AMD, Arm, and Microsoft, agentic AI could push CPU-to-GPU ratios from 1:4 to even1:1

In OCP APAC 2026, Tai AMD SVP of compute and enterprise AI said agents don't cut GPU demand but they just pile on a whole extra layer of orchestration, retrieval, and tool-calling work that runs on CPUs instead And the usual 1:4 CPU-to-GPU ratio could move toward 1:2 or even 1:1…

23
r/LocalLLaMA community 2d ago

RAG for regular users?

One of the reasons I got into local LLMs was the possibility of getting answers using my own documents and books (a few hundreds) instead of having to search through them manually. However since I'm not a data specialist or an engineer, RAG projects were too difficult for me,…

17
r/LocalLLaMA community 2d ago

It's the final countdown, baby! Qwen is out in just over 7 hours!

Historic event! We're ready! Google Translate, on the other hand, is not ready!   submitted by   /u/LegacyRemaster [link]   [comments]

11
r/LocalLLaMA community 2d ago

FYI: Muse Glimmer Chat Template Got Updated Recently

Source: https://huggingface.co/meta-models/Muse-Glimmer-30B/blob/main/chat_template.jinja Seems to be a deduplication. Not sure how it alters model performance but it did get updated FWIW. Side note: good orchestrator model, Meta!   submitted by   /u/TacticalRock [link]…

36
r/LocalLLaMA community 2d ago

RTX 6000 PRO price raised to $16,000 USD on the Nvidia website

  submitted by   /u/Norwood_Reaper_ [link]   [comments]

29
r/LocalLLaMA community 2d ago

What unique, custom QOL upgrades have you given your local agents?

Warning : Kinda long post. If you don't like reading, please skip for your own sanity. Also, I've got nothing to sell, just a tinkerer, so I just want to share ideas and learn from you guys too. When I recently started my agentic journey, I hit a number of walls, the first being…

4
r/LocalLLaMA community 2d ago

New Muse-Glimmer-30B SoTA Quants - hopefully a new lineup :)

Hey Folks, I've been making quants for a while - recently I took a short break to get into hardcore research (submitted my first EMNLP paper during it!). Along the way, I built up a little arsenal of quant-optim techniques: everything from novel, paper-pending tricks to some…

19
r/LocalLLaMA community 2d ago

Anthropic, OpenAI, Google, Meta, Microsoft, and Mistral all signed the EU Code of Practice on Transparency of AI-Generated Content

Even open source local models from these companies will be watermarking code and text since it's required by law.   submitted by   /u/Bestlife73 [link]   [comments]

19
r/LocalLLaMA community 2d ago

Toy project: a chat title model that fits in 5 MiB of ram

Not even sure if I'm allowed to post this, what with the "completely/primarily LLM generated copy" rule (the post itself is fine, but the repo/model I'm sharing definitely is, whoops) and the whole limit self-promotion thing, but it's just a toy I made that is trying to solve a…

13
r/LocalLLaMA community 2d ago

I tested the CMP170HX

Lots of rumor and misinfo bouncing around, so I put some of these old mining cards to the test. I used 4 of the 8GB cards, set to 64GB each. Lots of models fit entirely on a single card, and you can also run several small models at the same time on a single card, as long as…

15
r/LocalLLaMA community 2d ago

Can Gemma and Qwen models catch hallucinations by looking at their own logprobs?

Hi! I'm really obsessed with LLM hallucinations for the last 6 days 😭 I started by designing system prompts to attack hallucinations but failed, obviously. Now I tried reading logprobs and... I think when the model recalls its first fact in its chain of thought, before it has…

7
r/LocalLLaMA community 2d ago

We quantized DeepSeek V4 0731 and benchmarked it against popular quants on 8× RTX 5090

We converted the model from the original safetensors and found two issues. The first one made our quantization fail several times, the second one does not fail at all, it just quietly ruins the base 1) You must use the --no-lazy option, otherwise token_embd.weight will take on…

23
r/LocalLLaMA community 2d ago

366 t/s Qwen3.6 27B NVFP4 on v100s

These are single stream numbers Following on from my previous post about v100s ( here ) and inspired by this comment ( here ) I decided to work on kernels that allow for an extremely fast path for Nvfp4 weights on sm70 and almost free deep speculation on sm70 as well. Which…

19
r/LocalLLaMA community 2d ago

Local Benchmark : Muse Glimmer 30B vs Qwen 3.6 27B vs Gemma4 31B (and many other models and finetunes)

Needs a lot of requests compared to Qwen (almost twice) and Gemma (almost x3). Final score is fine, even though it is "not a coding model" https://wonderrico.github.io/local_llm_benchmark/benchmark-main.html more details on…

31
r/LocalLLaMA community 2d ago

I will be parting with my 4x Spark Cluster.

Laid off then my partner of 10 years said he's leaving, have to move, etc... I will post the r/hardwareswap link when I make it. I'm willing to add some incentive for r/LocalLLaMA folks. I will also add the super node configs and all the cool stuff that may not be apparent that…

26
r/LocalLLaMA community 2d ago

All the more reason not to use Closed Models ... Claude now officially "marks" AI-generated content ... steganographically, apparently ... and there are false positives already

  submitted by   /u/johnnyApplePRNG [link]   [comments]

31
r/LocalLLaMA community 2d ago

[llama.cpp PR #26608] Ling-3.0 support (unmerged)

aetherbird has done some great work getting Ling-3.0 to work in llama.cpp. The architecture is generally identical to deepseekv2. I recently added a microscopic 40 line PR to his that adds support for the Tiny model, works great. Using it for home assistant voice with decent…

23
r/LocalLLaMA community 2d ago

Ling-3.0-flash quant ladder on one DGX Spark: the whole thing sits in a 32 to 40 tok/s band

The interesting part of this one isn't the top number, it's how little distance there is between the top and the bottom of the ladder. Where it comes from: I work on Ling at inclusionAI, these aren't my numbers. sudoingX on X benched the full community GGUF ladder on his own DGX…

36
r/LocalLLaMA community 2d ago

Muse Glimmer 30B running locally in-browser with custom WebGPU kernels at ~25 tok/s on an M4 Max (same speed as llama.cpp)

  submitted by   /u/xenovatech [link]   [comments]

23
r/LocalLLaMA community 2d ago

DeepSeek V4 Flash 0731 at 27+ t/s decode on Strix Halo — Vulkan + DSpark full guide

Been benchmarking DSv4 Flash 0731 on a Flow Z13 (Ryzen AI MAX+ 395, Radeon 8060S / gfx1151, 128GB LPDDR5X) for the past week. Figured I'd share what actually works and what doesn't — there are a lot of gotchas on this hardware. Results Best client-side observation (bench-kv.sh…

7
r/LocalLLaMA community 2d ago

nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16

  submitted by   /u/TheRealMasonMac [link]   [comments]

21
r/LocalLLaMA community 2d ago

Encrypted reasoning from ClosedAI et al 100% recoverable

Interesting examples in the link Paper here: https://arxiv.org/abs/2608.09867 This is your prompt to go out and give us 10mil rows of Opus 5 traces on hf before they fix this workaround   submitted by   /u/Dany0 [link]   [comments]

21
r/LocalLLaMA community 2d ago

I built a weird, low-power llama.cpp server using an Intel N100 + RTX 5060Ti

Everything started with the sudden death of my old ASRock J1900. While looking for the perfect ITX replacement, I stumbled upon the Chinese CW-NAS-ADLN-K motherboard, which looked perfect on paper: Intel N100, DDR5, 6x SATA, 2x NVMe. The extra power allowed me to experiment more…

11
r/LocalLLaMA community 2d ago

Introducing Unsloth Desktop app

Hi LocalLlama, we're super excited to release Unsloth Desktop today! 🦥 It's the first desktop app that enables you to run and train models locally. Open-source. Available on Mac, Windows, and Linux Supports MLX, diffusion image/video models, audio models, and GGUF You can run…

19
r/LocalLLaMA community 2d ago

I put Gemma 4 E4B and E2B into an e-reader so I can ask my weird questions and share my thoughts in private directly in app.

Here's how it works in the app: Framework: Runs on LiteRT-LM (like Google's AI Edge). Models: Downloads either the E2B (~2.5 GB) or E4B (~3.6 GB) INT4 quantized models directly from ungated litert-community repos without API keys, tokens, or accounts. Execution & Memory:…

29
r/LocalLLaMA community 2d ago

CJK Manga/Manhwa/Manhua 150M OCR model (hayai-ocr-v2) outperforming PaddleOCR-VL-For-Manga

https://preview.redd.it/o7794vsszqih1.png?width=687&format=png&auto=webp&s=3b217db46fb51c517e103d12e2e2ca9813b7774f https://preview.redd.it/ci1as7nmzqih1.png?width=800&format=png&auto=webp&s=811e0b30de0f76ee548ef7701359674d8f8bf32a I trained a custom model with a custom decoder…

8
r/LocalLLaMA community 2d ago

nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 · Hugging Face

  submitted by   /u/coder543 [link]   [comments]

25
r/LocalLLaMA community 2d ago

The small open weight models are scarier in AI development

Imagine if your everyday laptop could run an AI model smart enough to take care of 90% of your work—totally private, lightning fast, and completely free of monthly fees. That is the exact tipping point of history where the AI bubble bursts completely. I bet this is the nightmare…

38
r/LocalLLaMA community 2d ago

Revision Prompting: Trades slow (decoded) output tokens for cheap (prefilled) input tokens.

TL;DR: If you re-run the same prompt whenever the input changes, try sending the old input/output plus a diff of the input, and ask the model for a patch to the output. You generate ~2-10x fewer output tokens, and the untouched parts of the output stay byte-identical. This…

22
r/LocalLLaMA community 2d ago

We even got a fgn manifesto!! Meta is on a run!

Zuck argues for releasing more open-weight models and invites governments to work with AI makers to test safety..who's I have yet to figure.   submitted by   /u/uhuge [link]   [comments]

6
r/LocalLLaMA community 2d ago

Tested in Coding: BF16 Muse Glimmer vs BF16 Qwen3.6 27B

I'm guessing that many people have been waiting for this comparison. For clarity, both models are running at full FP16 KV-cache. Due to VRAM limitations, Muse Glimmer is running full 262,144 context, whilst Qwen3.6 27B can only run at 147,500 context - full GPU offload in both…

28
r/LocalLLaMA community 2d ago

A friendly reminder that tomorrow is Qwednesday.

That will be all. Good luck falling asleep.   submitted by   /u/swagonflyyyy [link]   [comments]

10
r/LocalLLaMA community 3d ago

What can us 8 GB VRAM poors do?

I want to hook up a local model to Cline, but it seems the best model is still just Qwen 3.5 9B. Please can we have a Qwen 3.8 9B that gets close to Qwen 3.6 27B?   submitted by   /u/Aggravating-Push-207 [link]   [comments]

31
r/LocalLLaMA community 3d ago

Luth-2: New State-of-the-Art French Small Language Models

Hey everyone, Today we release Luth-2-0.8B and Luth2-2-2B , two non-reasoning models that set a new state of the art for French across a wide variety of tasks for their size 🚀 A few notable scores on French benchmarks compared to models 〜3 times their size: - Luth-2-2B scores…

31
r/LocalLLaMA community 3d ago

DeepSeek-V4-Flash acting as my Linux sysadmin

I'm very happy with some Linux admin tasks I'm throwing at a locally running DeepSeek. My request was simple, check why 'samples' folder is taking more and more space on one of the machines on my LAN, is this a syncthing problem? OpenCode used DeepSeek-V4-Flash to investigate,…

9
r/LocalLLaMA community 3d ago

Why have 8B-12B models been dropped?

I am a Macbook Pro M4 user with the 16GB of unified ram. The best model I have been able to run on LM Studio is Gemma4 12B QAT, this model is 66 days old. After that the next best thing LM studio suggests is Nemotron 3 Nano 4B and Qwen3.5 9B, which both are 147-161 days old. It…

37
r/LocalLLaMA community 3d ago

Rumored 50-series Super refresh bumps everything +50% VRAM

leaked Super specs have the 5070 Ti and 5080 going 16GB to 24GB and the 5070 to 18GB, thanks to the new 3GB GDDR7 modules. 24GB on a Ti-class card is actually the number people here have been waiting for. thats enough to run a lot of mid-size models locally at a decent quant…

25
r/LocalLLaMA community 3d ago

I ran Muse Glimmer @ 1M context - All tests passed.

Heeeey all! I just completed some fun tests with Muse Glimmer, I thought I'd let you know. In fact, the summary below was written by Muse itself! I ran a 2× DGX Spark cluster and got Meta's day-old Muse Glimmer 30B running the day after release — then pushed its context from the…

36
r/LocalLLaMA community 3d ago

Qwen 3.8-27b coming this week

Confirmed by the official Qwen account.   submitted by   /u/Bestlife73 [link]   [comments]

25

DeepSeek-V4-Pro-0813 is UP!

Qwen3.8-Max

Qwen3.8-2.4T-A95B Released

Qwen/Qwen3.8-2.4T-A95B · Released!

Qwen 3.8 2.4T is out , no 27b today RIP.

LiquidAI/LFM2.5-VL-3B · Hugging Face

Exact Qwen 3.8 27b release date and time

NVIDIA's Fastest Blackwell GPU, the 96 GB RTX PRO 6000, Now Costs $16,000, Almost Double Its Original Price

All your reasoning are belong to us

Hidden Reasoning from Claude and GPT are Decoded, and it is interesting

Intel LLM-Scaler ready with Muse Glimmer support, other LLMs & features

According to AMD, Arm, and Microsoft, agentic AI could push CPU-to-GPU ratios from 1:4 to even1:1

RAG for regular users?

It's the final countdown, baby! Qwen is out in just over 7 hours!

FYI: Muse Glimmer Chat Template Got Updated Recently

RTX 6000 PRO price raised to $16,000 USD on the Nvidia website

What unique, custom QOL upgrades have you given your local agents?

New Muse-Glimmer-30B SoTA Quants - hopefully a new lineup :)

Anthropic, OpenAI, Google, Meta, Microsoft, and Mistral all signed the EU Code of Practice on Transparency of AI-Generated Content

Toy project: a chat title model that fits in 5 MiB of ram

I tested the CMP170HX

Can Gemma and Qwen models catch hallucinations by looking at their own logprobs?

We quantized DeepSeek V4 0731 and benchmarked it against popular quants on 8× RTX 5090

366 t/s Qwen3.6 27B NVFP4 on v100s

Local Benchmark : Muse Glimmer 30B vs Qwen 3.6 27B vs Gemma4 31B (and many other models and finetunes)

I will be parting with my 4x Spark Cluster.

All the more reason not to use Closed Models ... Claude now officially "marks" AI-generated content ... steganographically, apparently ... and there are false positives already

[llama.cpp PR #26608] Ling-3.0 support (unmerged)

Ling-3.0-flash quant ladder on one DGX Spark: the whole thing sits in a 32 to 40 tok/s band

Muse Glimmer 30B running locally in-browser with custom WebGPU kernels at ~25 tok/s on an M4 Max (same speed as llama.cpp)

DeepSeek V4 Flash 0731 at 27+ t/s decode on Strix Halo — Vulkan + DSpark full guide

nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16

Encrypted reasoning from ClosedAI et al 100% recoverable

I built a weird, low-power llama.cpp server using an Intel N100 + RTX 5060Ti

Introducing Unsloth Desktop app

I put Gemma 4 E4B and E2B into an e-reader so I can ask my weird questions and share my thoughts in private directly in app.

CJK Manga/Manhwa/Manhua 150M OCR model (hayai-ocr-v2) outperforming PaddleOCR-VL-For-Manga

nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 · Hugging Face

The small open weight models are scarier in AI development

Revision Prompting: Trades slow (decoded) output tokens for cheap (prefilled) input tokens.

We even got a fgn manifesto!! Meta is on a run!

Tested in Coding: BF16 Muse Glimmer vs BF16 Qwen3.6 27B

A friendly reminder that tomorrow is Qwednesday.

What can us 8 GB VRAM poors do?

Luth-2: New State-of-the-Art French Small Language Models

DeepSeek-V4-Flash acting as my Linux sysadmin

Why have 8B-12B models been dropped?

Rumored 50-series Super refresh bumps everything +50% VRAM

I ran Muse Glimmer @ 1M context - All tests passed.

Qwen 3.8-27b coming this week