Deciphering Two Training Clocks in Grokking via Deep Linear Network Theory with Conditional ReLU Reduction 事件

Deciphering Two Training Clocks in Grokking via Deep Linear Network Theory with Conditional ReLU Reduction arXiv:2606.05863v1 Announce Type: cross Abstract: Grokking suggests that fitting the training data and learning a simple underlying rule may occur on different time scales. We formalize this phenomenon by separating the fast decay of the classification loss from the slower simplification of the learned representation, and we call the resulting pair of stopping times two training clocks. Fo