Routing by Analogy: kNN-Augmented Expert Assignment for Mixture-of-Experts 事件

Routing by Analogy: kNN-Augmented Expert Assignment for Mixture-of-Experts arXiv:2601.02144v2 Announce Type: replace Abstract: Mixture-of-Experts (MoE) architectures scale large language models efficiently by employing a parametric ``router'' to dispatch tokens to a sparse subset of experts. Typically, this router is trained once and then frozen, rendering routing decisions brittle under distribution shifts. We address this limitation by introducing kNN-MoE, a retrieval-augmented routing framew