MONA: Muon Optimizer with Nesterov Acceleration for Scalable Language Model Training 事件

MONA: Muon Optimizer with Nesterov Acceleration for Scalable Language Model Training arXiv:2605.26842v1 Announce Type: cross Abstract: The Muon optimizer has recently offered a promising alternative to AdamW for large language model training, leveraging matrix orthogonalization to produce geometry-aware updates. However, like all first-order methods, Muon can become trapped in sharp local minima. In this work, we present MONA, an optimizer that bridges Muon's orthogonalization framework with cu