Universal One-third Time Scaling in Learning Peaked Distributions 事件

Universal One-third Time Scaling in Learning Peaked Distributions arXiv:2602.03685v2 Announce Type: replace-cross Abstract: Training large language models (LLMs) is computationally expensive, partly because the loss exhibits slow power-law convergence whose origin remains debatable. Through systematic analysis of toy models and empirical evaluation of LLMs, we show that this behavior can arise intrinsically from the use of softmax and cross-entropy. When learning peaked probability distribution