Linearizing Vision Transformer with Test-Time Training 事件

Linearizing Vision Transformer with Test-Time Training arXiv:2605.02772v2 Announce Type: replace Abstract: While linear-complexity attention mechanisms offer a promising alternative to Softmax attention for overcoming the quadratic bottleneck, training such models from scratch remains prohibitively expensive. Inheriting weights from pretrained Transformers provides an appealing shortcut, yet the fundamental representational gap between Softmax and linear attention prevents effective weight tran