If MOEs have small experts (3B/4B/9B etc), then why can’t we have small expert models as a whole rather than one large model with multiple experts? Like Qwen3.6-3B Coding Expert or something
Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.
Forgive me naiveness, I’m a little lost. Why must a model be jack of all trades? Why can’t we have a model expert in a single thing?
[link] [comments]
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.