Causal Evidence of Stack Representations in Modeling Counter Languages Using Transformers 事件

Causal Evidence of Stack Representations in Modeling Counter Languages Using Transformers arXiv:2606.03398v1 Announce Type: new Abstract: Formal languages have proven to be effective conduits to understand the inner mechanisms of transformers. Past work has shown that transformers trained on next token prediction over counter languages learn representations consistent with an underlying stack structure. Beyond representational analysis, this paper investigates the causal role of these represent