On the "Induction Bias" in Sequence Models 事件

On the "Induction Bias" in Sequence Models arXiv:2602.18333v2 Announce Type: replace-cross Abstract: Despite the remarkable practical success of transformer-based language models, recent work has raised concerns about their ability to perform state tracking. In particular, a growing body of literature has shown this limitation primarily through failures in out-of-distribution (OOD) generalization, such as length extrapolation. In this work, we shift attention to the in-distribution implications