LoopMoE: Unifying Iterative Computation with Mixture-of-Experts for Language Modeling 事件

LoopMoE: Unifying Iterative Computation with Mixture-of-Experts for Language Modeling arXiv:2606.04438v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) and looped architectures scale models along two orthogonal axes, namely parameter capacity and effective depth. However, mainstream looped architectures rely on dense backbones that couple parameter count with per-token FLOPs, which makes it impossible to isolate the effect of iterative computation under matched budgets. To this end, we