Beyond Sinusoids: A Morlet Wavelet Framework for Transformer Positional Encoding 事件

Beyond Sinusoids: A Morlet Wavelet Framework for Transformer Positional Encoding arXiv:2606.01258v1 Announce Type: cross Abstract: Standard positional encodings for transformers - sinusoidal and rotary (RoPE) - treat every position as equally local: they encode where a token is, but not how far its positional influence should extend. We propose that the Morlet wavelet, which simultaneously minimises uncertainty in position and frequency, is the natural basis for positional encoding, and introdu