Multi-Rollout On-Policy Distillation via Peer Successes and Failures 事件

Multi-Rollout On-Policy Distillation via Peer Successes and Failures arXiv:2605.12652v2 Announce Type: replace-cross Abstract: Large language models are often post-trained with sparse verifier rewards, which indicate whether a sampled trajectory succeeds but provide limited guidance about where reasoning succeeds or fails. On-policy distillation (OPD) offers denser token-level supervision by training on student-generated trajectories, yet existing methods typically distill each rollout independ