Mitigating Staleness in Asynchronous Pipeline Parallelism via Basis Rotation 事件

Mitigating Staleness in Asynchronous Pipeline Parallelism via Basis Rotation arXiv:2602.03515v2 Announce Type: replace-cross Abstract: Asynchronous pipeline parallelism maximizes hardware utilization by eliminating the pipeline bubbles inherent in synchronous execution, offering a path toward efficient large-scale distributed training. However, this efficiency gain can be compromised by gradient staleness, where the immediate model updates with delayed gradients introduce noise into the optimiz