DTop-p MoE: Sparsity-Controlled Dynamic Top-p MoE for Foundation Model Pre-training 事件

DTop-p MoE: Sparsity-Controlled Dynamic Top-p MoE for Foundation Model Pre-training arXiv:2512.13996v2 Announce Type: replace Abstract: Sparse Mixture-of-Experts architectures are essential for scaling model capacity efficiently, yet the standard Top-$k$ routing imposes a rigid sparsity pattern that ignores the intrinsic variance in token difficulty and layer-specific computational needs. Top-$p$ routing is more adaptive because it selects experts until their cumulative routing probability reac