Llama.cpp recently introduced support for Programmatic Dependent Launch (PDL), which is a new feature in Nvidia GPUs (CC >= 90, not including ADA) such as Blackwell. (See PR 22522.)

In short, PDL enables more efficient execution of kernels and as a result better performance. So far, it's not enabled by default, if you don't know about it, you will likely miss it.

To enable PDL you need to build Llama.cpp with the '-D GGML_CUDA_PDL=ON' flag and it's not yet enabled for all kernels, there is likely more performance to be had once more kernels are enabled with PDL.

(To later disable PDL, if needed, do 'export GGML_CUDA_PDL=0' before starting llama.cpp)

Benchmarks

(All tests run with b9282 and results are best of two on an RTX Pro 4500 Blackwell 32GB.)

Conclusion

There is virtually no difference on pre-fill, however there is on average 5% to 6% performance boost on token generation based on above tests. According to the PR, somewhere between 4% and 10% improvement on token generation is expected.

As mentioned, this is not enabled by default when building, if you are on Blackwell, this is a free lunch and worth trying out.