Best llama cpp flags to run Deepseek-flash 0731
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
Hi all. These are my system specs: dual xeon e5 2696 v2 , 160gb DDR3 ram ECC(1600mhz), 3 gpus: 3060 12gb, p100 16gb, 3050 6gb. And a 400gb nvme sdd RAID0, 3000 mb/s. The model is Deepseek-flash-0731 UD_8_X_XL, loseless, 161gb. Now, I'm not too knowledgeable about llama cpp flags, I wish run it without mmap, because its so slow, and I believe it should fit in my system overall. There's also Dspark and MTP which could help with the speee, but do they work with llama? Any recommendations would help.
[link] [comments]
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.