SaluNet: Enabling Total Plasticity in Normalization-Free Deep Networks 事件

SaluNet: Enabling Total Plasticity in Normalization-Free Deep Networks arXiv:2606.02927v1 Announce Type: new Abstract: Normalization layers such as BatchNorm and LayerNorm have long been considered essential for stable training in deep networks. This work demonstrates that they can be fully replaced by a single learnable activation mechanism. We identify a plasticity suppression effect induced by standard normalization: learnable activation parameters rapidly lose adaptability when paired with