DOT-MoE: Differentiable Optimal Transport for MoEfication 事件

DOT-MoE: Differentiable Optimal Transport for MoEfication arXiv:2606.01666v1 Announce Type: cross Abstract: The scaling of Large Language Models (LLMs) has driven significant performance gains but created substantial challenges in inference efficiency. While Mixture of Experts (MoEs) architectures address this by decoupling model size from inference cost, training MoEs from scratch is often unstable and compute intensive. Conversion of pre-trained dense models into sparse MoEs has emerged as an