r/LocalLLaMA · · 1 min read

Best llama cpp flags to run Deepseek-flash 0731

Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.

Hi all. These are my system specs: dual xeon e5 2696 v2 , 160gb DDR3 ram ECC(1600mhz), 3 gpus: 3060 12gb, p100 16gb, 3050 6gb. And a 400gb nvme sdd RAID0, 3000 mb/s. The model is Deepseek-flash-0731 UD_8_X_XL, loseless, 161gb. Now, I'm not too knowledgeable about llama cpp flags, I wish run it without mmap, because its so slow, and I believe it should fit in my system overall. There's also Dspark and MTP which could help with the speee, but do they work with llama? Any recommendations would help.

submitted by /u/No_Farmer_495
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/LocalLLaMA