Energy-Gated Attention and Wavelet Positional Encoding: Complementary Inductive Biases for Transformer Attention 事件

Energy-Gated Attention and Wavelet Positional Encoding: Complementary Inductive Biases for Transformer Attention arXiv:2605.26355v1 Announce Type: cross Abstract: Standard transformer attention computes pairwise token similarity but treats all tokens as equally salient and all positions as equally local, regardless of the informational structure of the input. We identify two complementary inductive biases that standard attention lacks: energy salience (which tokens concentrate informational ene