The Many Faces of On-Policy Distillation: Pitfalls, Mechanisms, and Fixes 事件

The Many Faces of On-Policy Distillation: Pitfalls, Mechanisms, and Fixes arXiv:2605.11182v2 Announce Type: replace Abstract: On-policy distillation (OPD) and on-policy self-distillation (OPSD) have emerged as promising post-training methods for large language models, offering dense token-level supervision on trajectories sampled from the model's own policy. However, existing results on their effectiveness remain mixed: while OP(S)D has shown promise in system prompt and knowledge internalizati