A100 slow Qwen3.6-27B-FP8

Setting up a server for someone who has an A100 80GB, even though this doesn't natively support FP8 does 43tps decode sound too low for single request?

For comparison the exact same vllm config on my RTX 6000 PRO runs the same single request test at 130tps.

For 8 concurrent requests the A100 decodes at 177tps vs 509tps for the 6000.

--model Qwen/Qwen3.6-27B-FP8 --max-num-seqs 8 --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder --enable-prefix-caching --max-model-len auto --enable-chunked-prefill --kv-cache-dtype fp8 --speculative-config '{"method":"mtp","num_speculative_tokens":3}' 

Benchmarking with vllm bench (e.g. here with 1 concurrent request)

vllm bench serve \ --model "qwen3.6-27b-fp8" \ --tokenizer "Qwen/Qwen3.6-27B-FP8" \ --base-url "http://127.0.0.1:8000" \ --endpoint "/v1/completions" \ --dataset-name "random" \ --num-prompts 1 \ --random-input-len 1024 \ --random-output-len 4096 \ --trust-remote-code