Deepseek v4 flash - 100-150 faster t/s in prefill/pp.

You have two choices here (in order of pref):

1. Downgrade CUDA from 13.3 to 13.1 (skip 13.2 due to bugs) <- prefer this (thanks to u/fairydreaming for pointing this out)

2. Use this vibed fork that works with CUDA 13.3 https://github.com/vektorprime/working_ds4_speed

I was troubleshooting this yesterday with the nvidia profiler and some LLM help (https://www.reddit.com/r/LocalLLaMA/comments/1vcs7bl/ds4\_flash\_full\_model\_in\_offload\_600\_ts\_pp\_and/)

Here's some more info on #1 (quote from fairydreaming) "Downgrade your CUDA and recompile. Starting with 13.2 DeviceTopK is used for top-k instead of argsort, this turns PP rate to crap."

In short, DS4 Flash is spending a lot of time on things other than matrix multiplication.