Grokking or Glitching? How Low-Precision Drives Slingshot Loss Spikes 事件

Grokking or Glitching? How Low-Precision Drives Slingshot Loss Spikes arXiv:2605.06152v3 Announce Type: replace-cross Abstract: Deep neural networks exhibit periodic loss spikes during unregularized long-term training, a phenomenon known as the "Slingshot Mechanism." Existing work usually attributes this to intrinsic optimization dynamics, but its triggering mechanism remains unclear. This paper proves that this phenomenon is a result of floating-point arithmetic precision limits. As training e