Faster Synchronous On-Policy RL via Straggler-Aware Group Sizing 事件

Faster Synchronous On-Policy RL via Straggler-Aware Group Sizing arXiv:2606.02218v1 Announce Type: cross Abstract: Synchronous reinforcement learning methods such as Group Relative Policy Optimization (GRPO) provide stable and reproducible on-policy training, but they are highly vulnerable to stragglers, a single unusually long rollout can delay reward computation and parameter updates for the entire group. This problem becomes more severe as group size increases, creating a tension between the