Ngram and world knowledge - why are we just building a coding model?

This post is written by a human and I'd appreciate it if you treated it as such. Thanks.

So, I've been noticing a pretty clear interest in developing as good a coding and agentic tool-calling model as possible, especially at smaller sizes, sub-50 gigs. However, I'm finding that at least for my use of AI, if I really want to move away from big providers, I am going to require a model that has better world knowledge than the current offerings.

Qwen 3.8 27B is a truly fantastic model for tons and tons of stuff. It's highly intelligent, super good at designing applications and coding and working on my system. However, its world knowledge sucks​ compared to the frontier, especially at the Q4 quant that I have to run it at.

So, that leaves me with a question. With the new N-gram technology that we're seeing being baked into Qwen 3.8 Next and that presumably will run on future models, why can't a model be made that has a smaller set of intellectual capabilities but a greater amount of world knowledge? I understand that right now everyone is optimizing towards making as smart a model as possible fit into as small a space as possible. But why don't we leverage the SSD to give the model a lot of world knowledge and make models that are better at dealing with screenshots, multilingual capabilities, doing things like pixel art or answering physics questions?

ngram seems like the answer to the "can't fit in vram" question... Qwen 3.8 Next really opens my mind to the possibility that there could be a totally different and better paradigm for how these models are developed, at least for many use cases. Having a relatively smart model with a large amount of world knowledge might be better than having as smart a model as possible...

Not to mention that this would mean that a model's training cut off would become less relevant, because it could just be fashioned a new ngram.

Obviously, the main interest is in creating a model that can code as well as possible because that's what'll capture market share. But I am curious if there are any efforts into this kind of thing or if anybody has an idea on why these things aren't done more often.

Please tell me why I'm wrong, how I'm wrong, and in how many ways I'm wrong because I'm sure that that's all you really want to tell me, but at least I'll learn something, because as is obvious from this post I have no idea what I'm talking about.

Thanks have a good day :)

edit: I found this post and I guess it provides a lot of what I was asking:

https://www.reddit.com/r/LocalLLaMA/comments/1vzgtqf/ngram_vs_experts_explained/

edit2:

this one is even better, recconend reading. ty reddit suggestions:

https://www.reddit.com/r/LocalLLaMA/comments/1w0198r/no_engrams_wont_let_you_run_1t_models_locally_it/