Design Conditions for Intra-Group Learning of Sequence-Level Rewards: Token Gradient Cancellation 事件

Design Conditions for Intra-Group Learning of Sequence-Level Rewards: Token Gradient Cancellation arXiv:2604.13088v2 Announce Type: replace-cross Abstract: Reinforcement learning for multi-step reasoning with large language models (LLMs) typically relies on sparse terminal rewards, which creates a poorly conditioned credit-assignment problem: the final feedback is propagated uniformly across all intermediate decisions. This leads to high gradient variance, unstable training, and many ineffectiv