r/LocalLLaMA
500 articles archived · Visit source ↗ · RSS
-
r/LocalLLaMA community 1mo ago
Tested in Coding: Strata
This will be a potentially unpopular post - but it's the truth and grounded - so let's get to it. Hopefully you are familiar with my previous Tested in Coding series: https://www.reddit.com/r/LocalLLaMA/comments/1vvsokm/tested_in_coding_q8_k_xl_qwen38_27b_vs_bf16/…
5 -
-
-
llama.cpp on the stage
https://x.com/ggerganov/status/2107908265032229261   submitted by   /u/jacek2023 [link]   [comments]
26 -
-
llama : add a GPU cache for MoE experts kept in host memory by am17an · Pull Request #29887 · ggml-org/llama.cpp
Potentially big speedup for MoE models that don’t fully fit in VRAM. Are you GPU Poor? Show your speedups ;)   submitted by   /u/jacek2023 [link]   [comments]
37 -
-
-
LiquidAI/d1-3B · Hugging Face
  submitted by   /u/iamn0 [link]   [comments]
33 -
-
GPU Upgrade advice
Upgrade Question If we say the "budget max" is $1700-1800, and this could include upgrading to a Taichi motherboard: With a new motherboard , no need to bifurcate 1. Would you add a second 5060Ti (16GB)? Someone is selling one for around $500 locally; 2. Buy a used 7900 XTX…
35 -
-
-
-
Seeking upgrade advice
I got dual 7900xtx running on a z390 (pcie3 8x) running the 27b. I've been thinking upgrading the motherboard to a x570 (pcie4 8x) or a x870 (pcie5 8x) improving performance with TP, and then wait to buy medusa or spark with lpddr6 and 256GB (apple isn't an option for me).…
32 -
-
-
-
-
-
How far we’ve come
  submitted by   /u/like_a_tensor [link]   [comments]
25 -
-
-
-
-
-
-
-
-
-
-
Local AI ecosystem overview
Hey guys, it's Merve from Hugging Face! I've recently given a talk in a dev conference about llama.cpp + but also covering basic concepts like prefill vs decode, memory types, speculative decoding etc. you can use it if you feel like it and I appreciate if you can give…
33 -
-
-
-
54gb vram for 35$
Bought an old mining farm of a guy on avito (Russian eBay), guy had bought a garage a couple of years ago and it was sitting there for a while, found out it was a mining farm and put it up on there for sale for 5000 rub (~60 USD) since he wasn't sure if it works. I negotiated…
10 -
-
