Unlocking Feature Learning in Gated Delta Networks at Scale 事件

Unlocking Feature Learning in Gated Delta Networks at Scale arXiv:2606.04048v1 Announce Type: cross Abstract: Training and scaling Large Language Models demand enormous computational resources, motivating both efficient sub-quadratic architectures and principled hyperparameter tuning methods. While the Maximal Update Parametrization ($\mu$P) has enabled zero-shot hyperparameter transfer for standard Transformers, its extension to linear models, particularly those with structured state transitio