Research
MoE Routing Compared: DeepSeek, Qwen, Mixtral, Llama 4, gpt-oss
This article compares routing mechanisms across 10 open Mixture-of-Experts (MoE) models, detailing expert counts, top-k values, shared experts, gating functions, and load-balancing methods. It highlights trends like growing expert counts (from 8 to 384) and variations in top-k (from 1 to 8). The comparison is based on verified config.json files and technical reports, offering a practical reference for model deployment.
Mixture of ExpertsModel ArchitectureOpen Source Models