On Advantage Estimates for Max@K Policy Gradients 事件

On Advantage Estimates for Max@K Policy Gradients arXiv:2606.06080v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards is widely used for post-training reasoning models, but sparse outcome rewards make exploration difficult. A complementary approach is to optimize inference-time objectives such as pass@K and max@K directly, yet existing policy-gradient estimators for these objectives use different signals, baselines, and normalizations, making their relationships unc