Deepseek v4 Flash on 80 GB VRAM and 128 GB DDR4 RAM

I am using unsloth Q8 Deepseek v4 Flash. So far I am able to run properly with the following command

CUDA_DEVICE_ORDER=PCI_BUS_ID CUDA_VISIBLE_DEVICES=0,2,1 \ llamacpp/llama.cpp/build/bin/llama-server \ --model unsloth/DeepSeek-V4-Flash-GGUF/UD-Q8_K_XL/DeepSeek-V4-Flash-UD-Q8_K_XL-00001-of-00005.gguf \ --port 8001 --alias deepseek -lv 4 \ -ngl 99 \ --split-mode layer \ --tensor-split 30,30,20 \ --main-gpu 2 \ -ot "blk\.[0-3]\.ffn_(gate|up|down)_exps\.weight=CUDA0" \ -ot "blk\.[4-8]\.ffn_(gate|up|down)_exps\.weight=CUDA1" \ -ot "blk\.(9|1[0-5])\.ffn_(gate|up|down)_exps\.weight=CUDA2" \ -ot "ffn_(gate|up|down)_exps\.weight=CPU" \ -fa on \ --ctx-checkpoints 0 \ -c 262144 \ -b 4096 -ub 2048 \ --no-mmap \ --cache-type-k f16 --cache-type-v f16 \ --threads 16 \ --jinja \ --temp 1.0 --top-p 1.0 --min-p 0.0 \ --chat-template-kwargs '{"reasoning_effort":"high"}' \ --cache-ram 0 \ --parallel 1 

GPU 1 - CUDA 0 - RTX 3090 - 24 GB

GPU 2 - CUDA 1 - RTX 3090 - 24 GB

GPU 3 - CUDA 2 - RTX 5090 - 32 GB

I have tested the override tensor so that it provides the best placement of all the layers in the GPU and the MOE layers are moved to CPU. This also ensures optimal usage of the graphic cards as well.

So far the performance is better than Qwen 3.5 v2 on my coding tasks.

I am getting following speeds

10.28.822.853 I slot print_timing: id 0 | task 2332 | prompt eval time = 13348.73 ms / 506 tokens ( 26.38 ms per token, 37.91 tokens per second) 10.28.822.859 I slot print_timing: id 0 | task 2332 | eval time = 40260.52 ms / 365 tokens ( 110.30 ms per token, 9.07 tokens per second) 10.28.822.860 I slot print_timing: id 0 | task 2332 | total time = 53609.25 ms / 871 tokens 10.28.822.861 I slot print_timing: id 0 | task 2332 | graphs reused = 2641 10.28.822.940 I slot release: id 0 | task 2332 | stop processing: n_tokens = 870, truncated = 010.28.822.853 I slot print_timing: id 0 | task 2332 | prompt eval time = 13348.73 ms / 506 tokens ( 26.38 ms per token, 37.91 tokens per second) 10.28.822.859 I slot print_timing: id 0 | task 2332 | eval time = 40260.52 ms / 365 tokens ( 110.30 ms per token, 9.07 tokens per second) 10.28.822.860 I slot print_timing: id 0 | task 2332 | total time = 53609.25 ms / 871 tokens 10.28.822.861 I slot print_timing: id 0 | task 2332 | graphs reused = 2641 10.28.822.940 I slot release: id 0 | task 2332 | stop processing: n_tokens = 870, truncated = 0 

Please provide me any suggestions so that the token speed can increase.

I am using a very old motherboard (auros x570) and amd ryzen 3950x processor.

So, I think this is the best speeds for me without updating the system. Curious to hear from you.