Analyzing Stream Collapse in Hyper-Connections: From Diagnosis to Mitigation 事件

Analyzing Stream Collapse in Hyper-Connections: From Diagnosis to Mitigation arXiv:2606.03483v1 Announce Type: cross Abstract: Hyper-Connections (HC) replace the single Transformer residual stream with multiple streams, introducing a permutation symmetry over stream indices. We study how this symmetry is resolved in practice: whether streams specialize in a balanced way or exhibit dominant-stream usage. Using fine-grained diagnostics for HC-based language models, we trace how multi-stream repre