Is there a point where models just cannot get any smaller without losing intelligence?

DeepSeek V4 Flash got me thinking...

We keep seeing smaller models get way better. A model at a certain parameter count today can be much smarter than a model of the same size from a year or two ago. Better training, better data, better architectures, distillation, MoE, and all of that seem to let companies squeeze more intelligence into smaller models. But is there eventually a limit to this?

At some point, a model needs enough capacity to understand language, store knowledge across a huge number of subjects, reason through problems, write code, follow instructions, and generalize to things it has not seen before. So can we just keep shrinking models while maintaining the same level of intelligence?

Could a future 30B model actually match a current 300B or 700B model across everything? Not just on a few benchmarks, but in actual use across lots of different domains.

Could the same eventually happen with a 7B model? Or is there some minimum amount of capacity needed before the model starts losing knowledge, reasoning ability, or reliability?

I know parameter count is not a direct measurement of intelligence. MoE also makes this more confusing because a model can have hundreds of billions of total parameters while only using a small portion of them for each token. There is also a difference between total parameters, active parameters, memory usage, and actual inference compute.
I also do not think comparing parameters to neurons in the human brain is very useful. They are obviously not the same thing. Still, it makes me wonder whether there is some minimum amount of information or computation needed for something close to general intelligence.
Maybe we are not actually removing the cost either. Maybe we are just moving it somewhere else. A smaller model might require a much more expensive training run, synthetic data from larger models, distillation, longer reasoning time, retrieval, or external tools.

There is also the benchmark question. When a smaller model gets a similar benchmark score to a much larger one, does it really have the same overall capability? Or is it more optimized for the things we currently test?

Maybe it matches the larger model most of the time, but falls apart more often on rare knowledge, unusual prompts, long tasks, or problems that are very different from its training data.

My guess is that there is probably a minimum size for any specific level of capability, but better training and architectures keep pushing that minimum lower. I just wonder when the big improvements start slowing down.
Are we still early enough that models can keep getting dramatically smaller and smarter? Or are we getting close to the point where the easy gains are gone and the last 10 or 20 percent becomes extremely difficult?