Decoupling the "What" and "Where" With Polar Coordinate Positional Embeddings 事件

Decoupling the "What" and "Where" With Polar Coordinate Positional Embeddings arXiv:2509.10534v3 Announce Type: replace-cross Abstract: The attention mechanism in a Transformer architecture matches key to query based on both content -- the what -- and position in a sequence -- the where. We present an analysis indicating that what and where are entangled in the popular RoPE rotary position embedding. This entanglement can impair performance particularly when decisions require independent matche