On the Limits of Model Merging for Multilinguality in Pre-Training 事件

On the Limits of Model Merging for Multilinguality in Pre-Training arXiv:2605.25846v1 Announce Type: new Abstract: Endowing models with consistent multilingual performance can be achieved by mixing pre-training data, or post-training approaches such as language-specific model merging. In this work, we test whether merging can be applied to monolingually pre-trained models. We conduct a controlled study on the efficacy of mixed, merged, and monolingual pre-training setups. We find that while mon