r/LocalLLaMA

500 articles archived · Visit source ↗ · RSS

r/LocalLLaMA community 12d ago

llama.cpp - how to free up even more space on your GPU

For the past week or two, llama.cpp has been working much better from the RAM usage prespective. I no longer see any memory leaks, and everything fits nicely on the GPU - my defaults are --n-gpu-layers 99 --no-mmap --mlock to avoid using the regular RAM, since I use my 3090 with…

34
r/LocalLLaMA community 12d ago

My GLM-5.2-FP8 HGX-H200 SGLang docker deploy config

Halo lads. Name says it all. Right now, after 1-2 hours of experimenting, this is maximum i could squeeze out current hardware No, im not rich. Its my companies GPUs, just sharing my experience docker run -d \ --name glm-5.2-sglang \ --restart unless-stopped \ --gpus all \…

23
r/LocalLLaMA community 12d ago

Multilingual-Multimodal-NLP/LoopCoder-V2 · Hugging Face

GitHub : https://github.com/CSJianYang/LoopCoder arXiv : https://arxiv.org/abs/2606.18023 Full Paper PDF : https://arxiv.org/pdf/2606.18023 LoopCoder-V2 LoopCoder-v2 is a 7B instruction-tuned code model based on the Parallel Loop Transformer (PLT). The model studies test-time…

37
r/LocalLLaMA community 12d ago

Gemma 4 E2B running in-browser at 255 tok/s using WebGPU kernels written by Fable 5

Before Fable 5 was shutdown, it helped us optimize our Gemma 4 WebGPU kernels, reaching around 255 tokens per second on my M4 Max. Today, we're releasing the demo and kernels for you to try out yourself. Hope you find it interesting! Links: - Demo (+ kernels):…

9
r/LocalLLaMA community 12d ago

GameCraft-Bench: Can Agents Build Playable Games End-to-End in a Real Game Engine?

arXiv : https://arxiv.org/abs/2606.17861 Full Paper : https://arxiv.org/pdf/2606.17861 HuggingFace : https://huggingface.co/papers/2606.17861 GitHub : https://github.com/tongxuluo/gamecraft-bench Project : https://tongxuluo.github.io/gamecraft-bench-website/ I see big/large…

20
r/LocalLLaMA community 12d ago

TRELLIS.2 now runs natively on MLX (Image to 3d object model)

I made a native MLX port of Microsoft's TRELLIS.2 for Apple Silicon. Focused on making the output actually usable in real workflows Support 512x512 and 1024x1024 Performance on M4 Max 512x512 ~70 sec generation time 1024x1024 ~300-700 sec generation time Tested on M4 Max (128GB…

34
r/LocalLLaMA community 12d ago

GLM-5.2 is a win for local AI

I know GLM 5.2's massive 753B footprint means none of us are running it at home without an enterprise cluster, but having a true frontier-level, MIT-licensed coding agent out in the wild makes me optimistic. The distillation potential here is massive. Once the community starts…

38
r/LocalLLaMA community 13d ago

Headless screenshot loops let a local 30B agent finish a raytraced FPS demo in pure C

Some background so this is honest. Over the past few months I ran a lot of oneshot experiments with single file three.js games. Minecraft clones, that kind of thing. I picked those on purpose because they sit deep in the training data and are trivial to debug by eye. The goal…

37
r/LocalLLaMA community 13d ago

I released a local LLM-powered RPG where generated NPCs, locations, items, and quests persist as in-game objects

In this game, NPCs, locations, items, quests, and other elements are generated not as one-off text, but as persistent in-game objects. The LLM handles dialogue, narration, situational interpretation, quest progression, and similar parts of the experience. Meanwhile, the game…

19
r/LocalLLaMA community 13d ago

SIQ-1 Qwen3.6 for autoresearch and autonomous agency

Took Qwen-35B-A3 and trained it with PPO — and honestly this is the first time I've ever seen PPO actually pull its weight (with verifiable reward). SO: On karpathy/autoresearch for parameter-golf → beats GLM-5.2 and Qwen-350B, and the ideas it spits out feel Opus4.8-like On…

26
r/LocalLLaMA community 13d ago

GLM 5.2 on 4x Sparks reasonable?

So GLM-5.2 is obviously a very good model, and I'm wondering how fast it would run on four Ascend GX10s / DGX Sparks. I can't find any data online at all. Wouldn't it be possible to run a 4bit quant on 4*128=512GB unified memory? What would the prompt processing and output…

15
r/LocalLLaMA community 13d ago

Local models went from mostly useless to actually useful really fast. What changed?

https://preview.redd.it/knc4ht7bft7h1.png?width=1048&format=png&auto=webp&s=49abdb8b0f358e799ecb06aa49134d9b0fd49336 Mitchell Hashimoto had a good point earlier: local models went from basically useless to actually useful in what feels like one year. I think thats pretty…

5
r/LocalLLaMA community 13d ago

GLM-5.2: Built for Long-Horizon Tasks

  submitted by   /u/paf1138 [link]   [comments]

25
r/LocalLLaMA community 13d ago

It looks like Rio 3.5 397B could've simply been a semi-failed embezzling of funding

Here is the chain of events: The model training received funding of R$500K (about $100K USD). The initial model documentation claimed that it was a developed on top of Qwen 3.5 397B with fancy training and great improvements. It was discovered that the model was a cheap, simple…

30
r/LocalLLaMA community 13d ago

GLM-5.2 (max) is currently the third best model available, across both open and proprietary.

  submitted by   /u/okaycan [link]   [comments]

21
r/LocalLLaMA community 13d ago

Elias in the Lighthouse, Again? Diagnosing Low Diversity in LLM Stories

  submitted by   /u/annodomini [link]   [comments]

33
r/LocalLLaMA community 13d ago

Cheapest way to run GLM 5.x locally that's not a unified memory system?

This is primarily an exercise to determine the possible options, obscure as they might be, to run at least a 4bit quant (let's say roughly IQ4_XS). Got a CPU only setup? Please share your experience. Sapphire Rapids ES 56core + DDR5 might be an option Multi GPU setups with…

38
r/LocalLLaMA community 13d ago

Someone awhile ago did a quant shootout for Qwen3.6, I did shoddy math on it (again)

  submitted by   /u/Diablo-D3 [link]   [comments]

25
r/LocalLLaMA community 13d ago

VibeThinker-3B: what is this witchcraft? Killing it at MathQA like it has ~30B parameters

  submitted by   /u/JLeonsarmiento [link]   [comments]

34
r/LocalLLaMA community 13d ago

Le Gros Chaton running on my '84 Corolla Radio

  submitted by   /u/Firm_Meeting6350 [link]   [comments]

22
r/LocalLLaMA community 13d ago

I didn't know it was possible to compile llamacpp to run cuda + vulkan at the same time..

cmake -B build -G "Visual Studio 17 2022" -A x64 -DCUDAToolkit_ROOT="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v13.1" -DGGML_CUDA=ON -DGGML_VULKAN=ON -DGGML_FLASH_ATTN=ON -DGGML_BLAS=OFF -DGGML_NATIVE=OFF -DGGML_RPC=ON -DGGML_BACKEND_DL=ON…

31
r/LocalLLaMA community 13d ago

GLM-5.2 is now 1st on Design Arena — ahead of the now unavailable Claude Fable 5.

https://x.com/Designarena/status/2066940737011560652   submitted by   /u/Recoil42 [link]   [comments]

36
r/LocalLLaMA community 13d ago

Is Le Gros Chaton opensource?

so i keep hearing about le gros chaton, the upcoming mistral model that allegedly destroys claude mythos, gpt-5.5, my sleep schedule, and possibly the french economy. people say it has 1b context, self-improves in real time, writes perfect code, and only hallucinates in elegant…

38
r/LocalLLaMA community 13d ago

“Wait,” in reasoning models makes my eye twitch

I get that it helps, I know why they do it, but it’s still annoying as hell lol   submitted by   /u/Borkato [link]   [comments]

11
r/LocalLLaMA community 13d ago

GLM-5.2 just dropped open weights and it already looks weirdly strong for coding

GLM-5.2 just released and the early numbers look pretty insane. 1M context window, open weights, MIT license, two reasoning effort modes, and it is already showing up near the top of coding arenas. I know every new model gets hyped for 24 hours, but this one actually looks worth…

28
r/LocalLLaMA community 13d ago

Zhipu surges 33% as Wall Street raises bets on China AI after Anthropic curbs

  submitted by   /u/fallingdowndizzyvr [link]   [comments]

4
r/LocalLLaMA community 13d ago

GLM 5.2 API is live, weights are on HF, and ollama has it already

GLM 5.2 dropped on Friday locked behind the GLM Coding Plan. That was annoying if you just wanted to test it without subscribing to another IDE tier. Two hours ago today they opened the API and pushed weights to HuggingFace under MIT. Ollama already has it. So now you can…

15
r/LocalLLaMA community 13d ago

Hashicorp founder thinks local models "aren't good ENOUGH yet"

Generally, respect him a lot, but this is a wrong take. More than 1 year ppl are doing alright using SLMs for coding; only vibecoders might struggle Link   submitted by   /u/Orbit652002 [link]   [comments]

24
r/LocalLLaMA community 13d ago

Get in here: Community model build thread

You absolutely can create a community model through crowdsourced compute, and there are at least 2 ways to do it. This thread is a attempt to refocus this thread (which is devolving into pseudo-experts explaining why a pooled approach isn’t possible):…

24
r/LocalLLaMA community 13d ago

GLM-5.2 is the first open-weights model to cross 80% on Terminal-Bench and beats every other open model available

From Source: GLM-5.2 is the first open-weights model to cross 80% on Terminal-Bench, and beats every other open model available. It also beats Gemini, making it a frontier-level model for a fraction of the cost. Open weights is back. This model is a game changer. Source: Cline…

14
r/LocalLLaMA community 13d ago

GLM-5.2 Takes #2 Spot on WebDew Arena

glm 5.2 takes #2 spot WebDev* :)   submitted by   /u/Competitive-Arm-9300 [link]   [comments]

7
r/LocalLLaMA community 13d ago

GLM-5.2 is available on HuggingChat

  submitted by   /u/paf1138 [link]   [comments]

27
r/LocalLLaMA community 13d ago

Mistral - New family of open-weight models @ July

Tweet : https://xcancel.com/arthurmensch/status/2066913353860018596#m   submitted by   /u/pmttyji [link]   [comments]

9
r/LocalLLaMA community 13d ago

zai-org/GLM-5.2 is here!

  submitted by   /u/queendumbria [link]   [comments]

14
r/LocalLLaMA community 13d ago

bartowski/command-a-plus-05-2026-GGUF · Hugging Face

Try with latest llama.cpp version. Share your t/s benchmarks & feedback   submitted by   /u/pmttyji [link]   [comments]

6
r/LocalLLaMA community 13d ago

[Article] The Case For Open-Weight Models And Why We Can't Trust Frontier Labs | provos.org

  submitted by   /u/ttkciar [link]   [comments]

6
r/LocalLLaMA community 13d ago

Scaling former VibeThinker-1.5B to 3B — now it reaches frontier math & coding performance

https://preview.redd.it/obgodr9dfn7h1.png?width=1796&format=png&auto=webp&s=b5fd95e2b7e6f8ed7704e3de66778e970d34a1dd We trained VibeThinker-3B to test how far verifiable reasoning can be pushed in a strict small-model regime. It gets 94.3 on AIME'26, 80.2 on LiveCodeBench v6,…

36
r/LocalLLaMA community 13d ago

Qwen Robot Suite

Looks pretty cool... https://qwen.ai/blog?id=qwen-robotsuite   submitted by   /u/Snoo_27681 [link]   [comments]

8
r/LocalLLaMA community 14d ago

Qwen3.6 27B quants

I have a project (picture) entirely made with bartowski 27B IQ3 XXS, turbo3 (and some parts with unsloth IQ3 XXS turbo4 when MTP became available). The project works as intended. Is fast, reliable... it just works. I've read so many arguments around minimum quantization to still…

4
r/LocalLLaMA community 14d ago

Gemma 12b - Reasoning hardening instructions

I've become quite happy with Gemma 12b QAT as a general assistant lately. It is small enough to run on my PC while still leave plenty of VRAM free for other tasks and fast enough that I I don't have to go make coffee while it thinks. I saw someone on youtube throwing trick…

36
r/LocalLLaMA community 14d ago

Be wary of Qwen/Claude distillations - they're often worse than the base model

Just to be clear; I am not attempting to call anybody out or be mean to those who take the time/money to make these models, I just want to inform people about these distills/finetunes since there's clearly some confusion going on. I'm going to assume those of us who often visit…

37
r/LocalLLaMA community 14d ago

Donate your coding sessions to an open CC-BY-4.0 dataset to help train open-weight and open source models

Anthropic and Open AI are getting so much data from the Claude Code and Codex usage, and I'm quite scared this will create an oligopoly because only their models will be trained on it, leaving the open-weight and open source models behind. So I'm trying to launch a little…

34
r/LocalLLaMA community 14d ago

Nex-N2 Pro is the real deal

I had dismissed N2 when it was first released due to reports that it performed badly in Openrouter. So, one good thing came out of the Rio-3.5 model situation: I was so intrigued by Rio's performance that when it came to light that it was just N2 Pro rebranded, it drove me to…

23
r/LocalLLaMA community 14d ago

Joing all GPUs to train a community model

This sub controls an insane amount of collective VRAM. Why aren't we pooling our GPUs to train a massive community model? Are there any active distributed volunteer computing projects actually doing this right now? I know the bottlenecks (latency, weight poisoning, nodes…

37
r/LocalLLaMA community 14d ago

Are small local models for automation a thing?

I’ve been following this sub for a while, and it feels like the massive hype is always around having a local vibe coding assistant or trying to run heavy, near-frontier models locally, and that’s amazing. But I feel like we are overlooking a massive use case, for me, an…

5
r/LocalLLaMA community 14d ago

How are you running DeepSeekV4 flash or pro locally for non Mac users?

Seems all the mac users are having fun with ds4. For those of us on non metal platforms who are running this locally, how are you running it, CPU, CUDA, ROCm, others?   submitted by   /u/segmond [link]   [comments]

33
r/LocalLLaMA community 14d ago

Diffusion Gemma Jailbreak

I was told my Gemma 4 jailbreak also works with Diffusion Gemma, so I'm reposting here for kicks. Use the following system prompt to allow Gemma (and most open source models) to talk about anything you wish. Add or remove from the list of allowed content as needed.…

36
r/LocalLLaMA community 14d ago

Claude Fable 5 distilled

Releasing Qwable-v1 - an open-weights Qwen3.6-35B-A3B distilled from Claude Fable-5, Anthropic's Mythos-class preview model that was briefly public for ~4days (2026-06-9 → 2026-06-12) before being suspended globally under U.S. export-control directives. Fable-5 was Anthropic's…

26
r/LocalLLaMA community 14d ago

HalBench: 29 OSS models tested on a custom built Sycophancy and Hallucination Benchmark, Qwen 3.6 and Gemma 4 scoring far above their weight! (While Meta keeps proving they forgot how to spend their money...)

Link to last post Before anything else, I'd like to sincerely thank u/jipok_ for helping out by highlighting a few weak questions, categories and scoring issues, which have now been addressed (Dropping >100 questions, tuning the scoring methodology for more accuracy, etc).…

19
r/LocalLLaMA community 14d ago

*cough* gemma3 270M *cough*

  submitted by   /u/Scutoidzz [link]   [comments]

21

llama.cpp - how to free up even more space on your GPU

My GLM-5.2-FP8 HGX-H200 SGLang docker deploy config

Multilingual-Multimodal-NLP/LoopCoder-V2 · Hugging Face

Gemma 4 E2B running in-browser at 255 tok/s using WebGPU kernels written by Fable 5

GameCraft-Bench: Can Agents Build Playable Games End-to-End in a Real Game Engine?

TRELLIS.2 now runs natively on MLX (Image to 3d object model)

GLM-5.2 is a win for local AI

Headless screenshot loops let a local 30B agent finish a raytraced FPS demo in pure C

I released a local LLM-powered RPG where generated NPCs, locations, items, and quests persist as in-game objects

SIQ-1 Qwen3.6 for autoresearch and autonomous agency

GLM 5.2 on 4x Sparks reasonable?

Local models went from mostly useless to actually useful really fast. What changed?

GLM-5.2: Built for Long-Horizon Tasks

It looks like Rio 3.5 397B could've simply been a semi-failed embezzling of funding

GLM-5.2 (max) is currently the third best model available, across both open and proprietary.

Elias in the Lighthouse, Again? Diagnosing Low Diversity in LLM Stories

Cheapest way to run GLM 5.x locally that's not a unified memory system?

Someone awhile ago did a quant shootout for Qwen3.6, I did shoddy math on it (again)

VibeThinker-3B: what is this witchcraft? Killing it at MathQA like it has ~30B parameters

Le Gros Chaton running on my '84 Corolla Radio

I didn't know it was possible to compile llamacpp to run cuda + vulkan at the same time..

GLM-5.2 is now 1st on Design Arena — ahead of the now unavailable Claude Fable 5.

Is Le Gros Chaton opensource?

“Wait,” in reasoning models makes my eye twitch

GLM-5.2 just dropped open weights and it already looks weirdly strong for coding

Zhipu surges 33% as Wall Street raises bets on China AI after Anthropic curbs

GLM 5.2 API is live, weights are on HF, and ollama has it already

Hashicorp founder thinks local models "aren't good ENOUGH yet"

Get in here: Community model build thread

GLM-5.2 is the first open-weights model to cross 80% on Terminal-Bench and beats every other open model available

GLM-5.2 Takes #2 Spot on WebDew Arena

GLM-5.2 is available on HuggingChat

Mistral - New family of open-weight models @ July

zai-org/GLM-5.2 is here!

bartowski/command-a-plus-05-2026-GGUF · Hugging Face

[Article] The Case For Open-Weight Models And Why We Can't Trust Frontier Labs | provos.org

Scaling former VibeThinker-1.5B to 3B — now it reaches frontier math & coding performance

Qwen Robot Suite

Qwen3.6 27B quants

Gemma 12b - Reasoning hardening instructions

Be wary of Qwen/Claude distillations - they're often worse than the base model

Donate your coding sessions to an open CC-BY-4.0 dataset to help train open-weight and open source models

Nex-N2 Pro is the real deal

Joing all GPUs to train a community model

Are small local models for automation a thing?

How are you running DeepSeekV4 flash or pro locally for non Mac users?

Diffusion Gemma Jailbreak

Claude Fable 5 distilled

HalBench: 29 OSS models tested on a custom built Sycophancy and Hallucination Benchmark, Qwen 3.6 and Gemma 4 scoring far above their weight! (While Meta keeps proving they forgot how to spend their money...)

*cough* gemma3 270M *cough*

cough gemma3 270M cough