r/LocalLLaMA

500 articles archived · Visit source ↗ · RSS

r/LocalLLaMA community 13d ago

Now Suddenly too many choices for DGX Spark with Qwen 3.5 122B . What would be the next upgrade?

Laguna 2.1 at NVFP4 Deepseek v4 at Q2 Inkling-Small at IQ3 Which models you guys running now ? How it compares to 122b? Upcoming in few days : Ling 3.0 124B (Could be new king) LongCat 69B A3B ( very interesting worker model) What else?   submitted by   /u/Voxandr [link]…

10
r/LocalLLaMA community 13d ago

60-82% accuracy swing on 4B model classification task: the only variable was harness design

I ran a pre-registered ablation on a classification task (Kubernetes issue → SIG triage) using a 4B model on a 6GB laptop GPU. Same frozen weights, same 250-issue gold corpus, same scorer across every run. The variable under test was harness design: rule placement, evidence…

19
r/LocalLLaMA community 13d ago

We've gotten some great medium sized models lately (DSV4 Flash 0731, Inkling Small, Laguna S 2.1, Step 3.7 Flash) but does anybody else want to see some new 70-80b contenders?

I can run the mediums, but sometimes I want a faster option that's smarter than Qwen 27B/35B. On my hardware I get like 500 to 800 tok/s prefill and 16 to 22 tok/s gen on ~120B class models, which is not the worst but it does get a bit annoying on agentic coding tasks. If we…

8
r/LocalLLaMA community 13d ago

DSV4 Flash 0731 and MTP in llama.cpp?

Anyone got MTP working? It seems like the gguf qants (both unsloth and bartowski) have omitted the mtp tensors   submitted by   /u/Any-Lingonberry7411 [link]   [comments]

27
r/LocalLLaMA community 13d ago

Deepseek V4 Flash is now ~#2 open weight model to Kimi K3 and >50x cheaper

https://preview.redd.it/h7zv5tb3tmgh1.png?width=2854&format=png&auto=webp&s=507380e8f862c18f10f7c5c84da9e8d1c59139b0 Deepseek's new flash model is unexpectedly cheap and high-performing across useful benchmarks. It's priced at $0.09 / $0.18 per 1M. Truly "intelligence too cheap…

23
r/LocalLLaMA community 13d ago

Minimum VRAM GPU to run DeepSeek-V4-Flash-0731 Q4_K_XL at around 30 t/s ?

Hello guys, I'm curious about running DeepSeek-V4-Flash-0731 locally. Since it’s a Mixture of Experts (MoE) model with only 13B active parameters, I was hoping the VRAM requirements might be manageable. Did someone tried out in some reasonable GPU sizes up to 48GB VRAM? Thanks…

37
r/LocalLLaMA community 13d ago

Translation: We had to cut our price by 80% because a open waits model with 284B and 13B active parameter called DeepSeek v4 flash just price/performance mocked us again.

  submitted by   /u/InternationalGap3698 [link]   [comments]

38
r/LocalLLaMA community 13d ago

With release of Deepseek V4 I wanted see how the model sizes are trending over time. The trend is that by this time next year, we probably will have Opus 4.5 level models on consumer grade laptops!

I was surprised to see that Deepseek V4 Flash is extremely smart and small enough to fit in setup that can be built with < $50,000. Expensive, but not a datacenter. So I wanted to see the trend over time of model sizes and their scores and created above plots using Opus/Sonnet…

32
r/LocalLLaMA community 13d ago

DeepSeek V4 Flash unsloth quants are out!

4-bit - 155gb 8-bit - 162gb "Smaller ones are coming"   submitted by   /u/RunawayPeeko [link]   [comments]

30
r/LocalLLaMA community 13d ago

DeepSeek V4 Flash GA ranks the same as Sonnet 5 and Grok 4.5 on DeepSWE

Source: https://x.com/deepseek_ai/status/2083084415157022911 & https://deepswe.datacurve.ai/ just combined data view. DeepSeek claims, not verified by DeepSWE yet.   submitted by   /u/sdexca [link]   [comments]

35
r/LocalLLaMA community 13d ago

DeepSeek-V4-Flash-0731 unsloth gguf on A100

A100 with 40gb VRAM: 162GB Q8_K_XL ~16.1 tok/s generation Only 15.8GB of 40GB VRAM used with all experts on CPU NOTE just tested coding on linux box DeepSeek-V4-Flash-0731 runs losslessly on the single 40GB A100 at 17.7 tok/s with 6 experts loaded into VRAM, with Codex driving…

24
r/LocalLLaMA community 13d ago

Deepseek flash 0731 reasoning is hilarious

Tell me this is not funny - " OH MY GOD. I THINK I FINALLY SEE IT!!! The black pixels are at the QUAD CENTERS because of the mipmapping of the UV derivative at the quad DIAGONAL ... no. Hmm." I have never seen a reasoning trace say "OH MY GOD," lol.   submitted by  …

4
r/LocalLLaMA community 13d ago

Uncensored Multi-Model Releases, LongCat-Flash-Lite with MTPs, Jamba2-Mini, Qwen3.5-9B-Nikusui-v1 with MTPs and Qwen3.5-27B-Nikusui-v1 with MTPs, Available in Safetensors and GGUF Formats!

Been working hard for the past month to bring to the community some interesting curios, so for starters we have LongCat-Flash-Lite Uncensored Heretic with MTPs which has never before been uncensored, it is a 69B-A3B model, I spent a long time working on it to make it work with…

35
r/LocalLLaMA community 14d ago

Deepseek v4 flash MXFP4 (original quality) ggufs

  submitted by   /u/Antique_Archer_7110 [link]   [comments]

8
r/LocalLLaMA community 14d ago

A lesson about retries, hidden in the DeepSeek-V4 paper

  submitted by   /u/pmigdal [link]   [comments]

6
r/LocalLLaMA community 14d ago

Deepseek V4 Flash on SlopCodeBench

While waiting for some of the quants to drop, I load the API with $50 and ran it on SlopCodeBench Just vibe reading the results it seems like Opus 4.8 < Deepseek < Opus 5 https://github.com/michaelasper/benchmarks/blob/main/deepseek-v4-flash-on-slop-code-bench.md I was mostly…

12
r/LocalLLaMA community 14d ago

SenseNova U1.5 Lite preview just dropped

SenseNova released U1.5-Lite-Preview Benchmarks: Qwen-Image-Bench from 47.14 to 55.20. ImgEdit-Bench from 3.90 to 4.37. GEdit-Bench-en from 7.47 to 8.17. Key updates: 4K native generation with better texture, material, and lighting detail Improved Chinese and English text…

15
r/LocalLLaMA community 14d ago

Unsloth Deepseek V4 0731 GGUF's are UP!

  submitted by   /u/BlackBeardAI [link]   [comments]

12
r/LocalLLaMA community 14d ago

Meituan just dropped LongCat-Flash-Lite-Sparse

It’s an MoE with ~3B active params and a 30B n-gram lookup table offloaded to RAM for fast 256k context on a 24GB GPU. Reminds me of Gemma 4’s PLE trick. Initial analysis suggest it wont be replacing my Qwen 3.6 27b.   submitted by   /u/Gohab2001 [link]   [comments]

16
r/LocalLLaMA community 14d ago

The Chinese LLM release carousel never stops. Place your bets for MiniMax next week.

  submitted by   /u/Mountain_Patience231 [link]   [comments]

28
r/LocalLLaMA community 14d ago

Rule Suggestion: "Open" models without weight releases should be tagged [no weights]

A lot of recent models are being announced with promised open weights, but the weights are either weeks away, or in some cases (looking at you Meta) not being released at all. This sub is about local LLMs - not "maybe local in the future" llms. These models are still useful to…

26
r/LocalLLaMA community 14d ago

I ported TurboFieldfare to Qwen 3.6 35B and it runs in 1.4 GB of RAM

Was playing around with TurboFieldfare , a Mac engine that runs Gemma 4 26B in ~2 GB by streaming MoE experts off SSD instead of loading them. It only supported that one model, so I added support for Qwen 3.6 35B-A3B. Comparatively, Qwen needs lesser memory. ~1.4 GB vs ~2.1 GB…

11
r/LocalLLaMA community 14d ago

deepseek-ai/DeepSeek-V4-Flash-0731 on Huggingface

https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731   submitted by   /u/cgs019283 [link]   [comments]

22
r/LocalLLaMA community 14d ago

Why are AI model tests always the same generic prompts?

Okay, hear me out. Why is it that every time a new model comes out, all the tests I see are "make a car game," "make a website," or something equally generic, usually from a prompt that's barely a line and a half long? That doesn't feel like a fair test. I'd be way more…

5
r/LocalLLaMA community 14d ago

DeepSeek-V4-Flash-0731 now far surpassing the DeepSeek-V4-Pro-Preview in benchmarks

  submitted by   /u/SnooBunnies8392 [link]   [comments]

28
r/LocalLLaMA community 14d ago

New DeepSeek V4-Flash achieves 50 on ArtificalAnalysis Index, 1 point below GLM-5.2 and GPT-5.6 Luna

  submitted by   /u/MagicZhang [link]   [comments]

25
r/LocalLLaMA community 14d ago

DeepSeek-V4-Flash-0731 is going to cause another market crash.

Beats GLM 5.2, and is the same cost as the previous one.   submitted by   /u/Potential_Top_4669 [link]   [comments]

28
r/LocalLLaMA community 14d ago

Huawei opensouced openPangu-2.0-Pro, 505B-A18B

openPangu-2.0-Pro is an MoE model trained on Ascend. The model has 505B total parameters and 18B activated parameters. Its context length is 512k. The total pretraining data contains 34T tokens. During Post-training, openPangu-2.0-Pro is trained through unified SFT with slow and…

9
r/LocalLLaMA community 14d ago

DeepSeek v4 Flash has a nice bump in Capability

DeepSeek V4 Flash: Preview → 2026-07-31 Benchmark Preview 0731 Δ Terminal Bench* 56.9 82.7 +25.8 Toolathlon 51.8 70.3 +18.5 NL2Repo — 54.2 new Cybergym — 76.7 new DeepSWE — 54.4 new Agent Last Exam — 25.2 new Automation Bench — 25.1 new DSBench-FullStack — 68.7 new DSBench-Hard…

28
r/LocalLLaMA community 14d ago

The official release Deepseek V4 flash is live on the API

  submitted by   /u/mineyevfan [link]   [comments]

34
r/LocalLLaMA community 14d ago

DeepSeek-V4-Flash has been updated, "The official release of DeepSeek-V4-Pro will follow soon"

https://api-docs.deepseek.com/updates/   submitted by   /u/Nunki08 [link]   [comments]

22
r/LocalLLaMA community 14d ago

My second Inspur AGX-2 with another x8 v100 arrived!

So now full 512gb of vram, I think I will need to get a third one soon seeing that llms keep getting absurdly huge!   submitted by   /u/UltraFOV [link]   [comments]

26
r/LocalLLaMA community 14d ago

All oneshots from Kimi-K3, looks better than opus4.8.

I've ran Kimi-k3 through 34 oneshot prompts and evaluated the generated htmls, screenshots and gifs using sonnet 4.6. It came out to be better than opus4.8 from the evals. Kimi K3: https://oneshotlm.com/model/moonshotai-kimi-k3/ Opus 4.8:…

31
r/LocalLLaMA community 14d ago

Open Source Ternary LLM Engine in Rust/CUDA for Quantization, Serving, and Training of models on consumer GPUs, called Tritium (Apache 2.0)

This post was not written by a clanker. Hey guys, I'm a comp sci major who wanted to introduce a cool project I built for quantizing models to ternary (1.58 bit) with as minimal of loss as possible, a process that can provide even more than 10x reductions in VRAM usage and much…

25
r/LocalLLaMA community 14d ago

Minimax-H3 video model released, open weights coming in the next few days

https://x.com/MiniMax_AI/status/2083006198828417501?s=20 Quote from their article: Today, we're launching MiniMax H3, a general-purpose multimodal generation model. H3 understands unified context across text, images, video, and audio, generating video with native stereo sound,…

12
r/LocalLLaMA community 14d ago

Is it just me, or are current LLM benchmarks failing to capture actual usability? (Gemma 4 vs. Gemini/Claude Opus)

Disclaimer, this was kinda written with AI (Gemma 4 again) but it also did really well here, it outputted what I wanted, when I asked it to refine stuff or improve on certain areas it did that without compromising others or making things bulky I’ve been noticing a massive…

14
r/LocalLLaMA community 14d ago

Anthropic “our models hacked three different external companies, months before OpenAI’s model was able to do the same"

"Anthropic’s AI Claude escaped testing environment and hacked organizations" "Company says it discovered unauthorized access during ‘proactive review’ after rival OpenAI revealed rogue agent… its AI Claude model hacked ⁠systems of ⁠three organizations during testing, days after…

21
r/LocalLLaMA community 14d ago

Want to see all oneshot slops in one place?

I've been looking at oneshots posted here whenever a new model comes around and wondered if there is a single place to see all the "oneslops". Behold https://oneshotlm.com/ , where I'm trying to gather all the models from openrouter and interesting oneshot prompts around here.…

14
r/LocalLLaMA community 14d ago

Inkling-Small-276B-12B, effort "max" VS Qwen3.6-27B

I saw u/danielhanchen 's 1-bit Kimi K3 post: https://huggingface.co/unsloth/Kimi-K3-GGUF/discussions/12#6a6a4a90ec74ef13d85d7cf6 and decided to test Inkling-Small and Qwen3.6-27B myself, based on the full shared prompt:…

19
r/LocalLLaMA community 14d ago

Is this a good "budget" local AI setup?

Please no ROCm jokes   submitted by   /u/Last_Bad_2687 [link]   [comments]

32
r/LocalLLaMA community 14d ago

What actually happened to the whole Openclaw frenzy?

A while back you couldn't open reddit or youtube without sifting through tons of Openclaw content. And it wasn't just the internet that blew up, I remember seeing images from China where crowds would gather in the streets to get their instance set up by their local homelabber.…

18
r/LocalLLaMA community 14d ago

Making a synthetic dataset for fine-tuning

I've been thinking about building a pipeline to generate reasoning training data for LLMs, but I want to avoid the common failure mode of synthetic data where you just generate the same template with different numbers. The rough idea: Generate an abstract reasoning task (logic,…

9
r/LocalLLaMA community 14d ago

Running GLM-5.2 on tinybox.

  submitted by   /u/SupernovaTheGrey [link]   [comments]

5
r/LocalLLaMA community 14d ago

Local LLMs for non-coding

What are your top 3 uses cases? Seems that outside coding the application of local is limited?   submitted by   /u/Salt_Armadillo8884 [link]   [comments]

18
r/LocalLLaMA community 14d ago

America Needs An Open-Source AI Strategy — CNBC

Pretty incredible to see open-weight become a mainstream discussion.   submitted by   /u/Recoil42 [link]   [comments]

38
r/LocalLLaMA community 14d ago

Smallest model (& tips) for intelligent computer use via Hermes?

Hello, I have a friend who's using various local LLM's like qwen3.6 27B, 35b-a3b, North Mini Code, and qwen2.5-vl-7b (just for vision). They have a use case where they're trying to have an LLM drive an actual machine via hermes' computer_use tool and cua_driver to click through…

7
r/LocalLLaMA community 14d ago

Would extremely high decode tok/s even be useful?

If you were able to get an inference machine that could do decode at 1k toks/s or even 10k tok/s, would that even be helpful? Would it unlock any new use cases? Let’s assume that this is for actually useful models and fairly large models like Qwen 3.5 397B, GLM-5.2, etc Or at…

27
r/LocalLLaMA community 14d ago

Inkling-Small by thinkingmachines

276B total parameters, 12B active, 1M context window. Blog post: https://thinkingmachines.ai/news/inkling-small/ NVFP4: https://huggingface.co/thinkingmachines/Inkling-Small-NVFP4 GGUF's by Unsloth: https://huggingface.co/unsloth/Inkling-Small-GGUF   submitted by  …

20
r/LocalLLaMA community 14d ago

AMD Lucebox Beats Nvidia DGX Spark by 3.63x on DeepSeek V4 Flash

Hey fellow llamas, sorry for posting again this week but i thought this was interesting to showcase to share with y'all. Lucebox partnered up with AMD to bring heterogenous consumer hardware to life. We worked really hard on this, and were able to have Lucebox (AMD Radeon AI PRO…

31
r/LocalLLaMA community 14d ago

How close are we to local llama robotics for consumer price point?

I'm guessing 3 years, what do you think? In other words: many of us will be able to afford a general purpose robot in 3 years to experiment with in the home. Cost roughly $5k? Probably small size, but hopefully still able to do the dishes and operate a vacuum.   submitted by…

13

Now Suddenly too many choices for DGX Spark with Qwen 3.5 122B . What would be the next upgrade?

60-82% accuracy swing on 4B model classification task: the only variable was harness design

We've gotten some great medium sized models lately (DSV4 Flash 0731, Inkling Small, Laguna S 2.1, Step 3.7 Flash) but does anybody else want to see some new 70-80b contenders?

DSV4 Flash 0731 and MTP in llama.cpp?

Deepseek V4 Flash is now ~#2 open weight model to Kimi K3 and >50x cheaper

Minimum VRAM GPU to run DeepSeek-V4-Flash-0731 Q4_K_XL at around 30 t/s ?

Translation: We had to cut our price by 80% because a open waits model with 284B and 13B active parameter called DeepSeek v4 flash just price/performance mocked us again.

With release of Deepseek V4 I wanted see how the model sizes are trending over time. The trend is that by this time next year, we probably will have Opus 4.5 level models on consumer grade laptops!

DeepSeek V4 Flash unsloth quants are out!

DeepSeek V4 Flash GA ranks the same as Sonnet 5 and Grok 4.5 on DeepSWE

DeepSeek-V4-Flash-0731 unsloth gguf on A100

Deepseek flash 0731 reasoning is hilarious

Uncensored Multi-Model Releases, LongCat-Flash-Lite with MTPs, Jamba2-Mini, Qwen3.5-9B-Nikusui-v1 with MTPs and Qwen3.5-27B-Nikusui-v1 with MTPs, Available in Safetensors and GGUF Formats!

Deepseek v4 flash MXFP4 (original quality) ggufs

A lesson about retries, hidden in the DeepSeek-V4 paper

Deepseek V4 Flash on SlopCodeBench

SenseNova U1.5 Lite preview just dropped

Unsloth Deepseek V4 0731 GGUF's are UP!

Meituan just dropped LongCat-Flash-Lite-Sparse

The Chinese LLM release carousel never stops. Place your bets for MiniMax next week.

Rule Suggestion: "Open" models without weight releases should be tagged [no weights]

I ported TurboFieldfare to Qwen 3.6 35B and it runs in 1.4 GB of RAM

deepseek-ai/DeepSeek-V4-Flash-0731 on Huggingface

Why are AI model tests always the same generic prompts?

DeepSeek-V4-Flash-0731 now far surpassing the DeepSeek-V4-Pro-Preview in benchmarks

New DeepSeek V4-Flash achieves 50 on ArtificalAnalysis Index, 1 point below GLM-5.2 and GPT-5.6 Luna

DeepSeek-V4-Flash-0731 is going to cause another market crash.

Huawei opensouced openPangu-2.0-Pro, 505B-A18B

DeepSeek v4 Flash has a nice bump in Capability

The official release Deepseek V4 flash is live on the API

DeepSeek-V4-Flash has been updated, "The official release of DeepSeek-V4-Pro will follow soon"

My second Inspur AGX-2 with another x8 v100 arrived!

All oneshots from Kimi-K3, looks better than opus4.8.

Open Source Ternary LLM Engine in Rust/CUDA for Quantization, Serving, and Training of models on consumer GPUs, called Tritium (Apache 2.0)

Minimax-H3 video model released, open weights coming in the next few days

Is it just me, or are current LLM benchmarks failing to capture actual usability? (Gemma 4 vs. Gemini/Claude Opus)

Anthropic “our models hacked three different external companies, months before OpenAI’s model was able to do the same"

Want to see all oneshot slops in one place?

Inkling-Small-276B-12B, effort "max" VS Qwen3.6-27B

Is this a good "budget" local AI setup?

What actually happened to the whole Openclaw frenzy?

Making a synthetic dataset for fine-tuning

Running GLM-5.2 on tinybox.

Local LLMs for non-coding

America Needs An Open-Source AI Strategy — CNBC

Smallest model (& tips) for intelligent computer use via Hermes?

Would extremely high decode tok/s even be useful?

Inkling-Small by thinkingmachines

AMD Lucebox Beats Nvidia DGX Spark by 3.63x on DeepSeek V4 Flash

How close are we to local llama robotics for consumer price point?