Gave a try to Exllamav3 and it's great!

Following this post I decided to try GLM 5.3 Flash on a 8x3090 setup and I can now run a Q4 with surprising speed; 700tk/s prefill & 42tk/s decoding! (lcp & vllm do not allow me to get that).

Was afraid about quality but > 30m tokens with DSH and no issue (did not test vision yet, but looks supported).

Just to say that I am really grateful to Turboderp and we should really support as much as possible others projects even if they do not comply with all our needs yet and not rely only on the big guys.