PC Layer: Polynomial Weight Preconditioning for Improving LLM Pre-Training 事件

PC Layer: Polynomial Weight Preconditioning for Improving LLM Pre-Training arXiv:2606.06470v1 Announce Type: cross Abstract: We propose a preconditioning (PC) layer, a weight parameterization via polynomial preconditioner that ensures stable weight conditioning throughout LLM training. The PC module reshapes the singular-value spectrum of weight matrices via low-degree polynomial preconditioning. After training, the preconditioned weights can be merged back into the original architecture, incur