Not All Transitions Matter: Evidence from PPO 事件

Not All Transitions Matter: Evidence from PPO arXiv:2605.24071v1 Announce Type: cross Abstract: Training a reinforcement learning agent on-policy means collecting fresh experience at every update, and that experience comes with a hidden problem. Each state in a rollout is the direct output of the previous one, causally chained together by the agent's own actions. Because of this, consecutive transitions are never truly independent. They carry overlapping information, and the gradient signal the