GPU Power Efficiency Tips and Tricks

Ran a quick test sweep across the frequency range of the P100s using Darwin-36B (Qwen 3.6 35B-A3B sort of). Not a comprehensive test. Feel free to share your power efficiency tips, maybe we make a bigger thread as we explore ways to improve our efficiency, not just raw tps.

Try it:

sudo nvidia-smi -pm 1 -ac 715,1063 = 84% of the speed at 67% of the power (+25% tokens/joule). Resets on reboot — add it to your startup script. Report your results please!

Also: capping to 150W (-pl 150) cost us 0.3% on this workload — the caps almost never bind during layer-split serving, so they're free thermal insurance.

WTF AM I TALKING ABOUT?:
Slow down your card and your performance a little bit, save a disproportionate amount of energy. Classic overclocking turned on it's head.

The last 265 MHz costs 53 Watts (enough to light a few rooms) and only buys 16% more speed. That'd be the trade you're refusing.