Expert-Aware Causal Tracing of Factual Recall in Sparse MoE Language Models 事件

Expert-Aware Causal Tracing of Factual Recall in Sparse MoE Language Models arXiv:2606.03780v1 Announce Type: new Abstract: Causal tracing of factual recall has been studied predominantly in dense transformer language models, where interventions localize information flow to layers or feed-forward modules. Sparse mixture-of-experts (MoE) language models introduce a sharper question: when a factual prediction is mediated by a routed MoE block, which routed expert contributions matter? We formulat