r/LocalLLaMA · · 1 min read

Hoping for Optimized Smarter Upcoming Models .... Like DeepSeek-V4.1-Flash( KVCache + Engram) in Small/Medium/Big sizes

Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.

Hoping for Optimized Smarter Upcoming Models .... Like DeepSeek-V4.1-Flash( KVCache + Engram) in Small/Medium/Big sizes

It's still a dream for many folks to run medium size(30B range) models @ Q8 with Unquantized KVCache (256K Context) on their GPUs.

It would be awesome to have DeepSeek-V4.1-Flash's KVCache + Engram for all Upcoming models. Even for big models.

Engram - Heard that approximately 1/3-1/2 of Model size. Might come in different size range too. So 10-15 GB for 30B models.

Here few models with approximate numbers. Current models in Odd rows & Future/Fictional models in even rows(Bold). I just put 1GB for 256K context below though DeepSeek-V4.1-Flash takes only same 1GB for 1 million context.

Model Model Size 256K KVCache F16 MTP Vision Total GB
Qwen3.8-27B-Q8 29 16 1 1 47
Qwen4.0-27B-Q8 29 1 1 1 32
Qwen3.8-27B-Q4_K_M 17 16 1 1 35
Qwen4.0-27B-Q4_K_M 17 1 1 1 20
Muse-Glimmer-30B-Q8 30 16 1 1 48
Muse-Glimmer-2-30B-Q8 30 1 1 1 33
Gemma-4-31B 33 16 1 1 51
Gemma-5-31B 33 1 1 1 36
Qwen3.6-35B-A3B-Q4_K_M 23 6 1 1 31
Qwen4.0-35B-A3B-Q4_K_M 23 1 1 1 26
Gemma-4-26B-A4B-Q8 27 6 1 1 35
Gemma-5-26B-A4B-Q8 27 1 1 1 30

Possibly there might be few more things(Please share those) to keep these number down. So I think 32GB VRAM is more than good enough for Upcoming (Optimized Smarter) Models. RAM is enough for Engram.

By above logic(based on Qwen4.0-27B), people could run Q4 of 54B models with same 32GB VRAM.

Maybe next year onwards, inventions could make 24GB enough for similar size models.

submitted by /u/pmttyji
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/LocalLLaMA