Gradient Descent with Large Step Size Restores Symmetry in Deep Linear Networks with Multi-Pathway 事件

Gradient Descent with Large Step Size Restores Symmetry in Deep Linear Networks with Multi-Pathway arXiv:2606.05219v1 Announce Type: cross Abstract: Recent analyses of multi-pathway Deep Linear Networks use Gradient Flow to predict a "winner-takes-all" specialization in which path symmetry breaks and each feature concentrates in a single pathway. In this work, we show that discrete Gradient Descent (GD) with a large step size tells a different story. We prove that single-path solutions are shar