Unleashing Implicit Rewards: Prefix-Value Learning for Distribution-Level Optimization 事件

Unleashing Implicit Rewards: Prefix-Value Learning for Distribution-Level Optimization arXiv:2604.13197v2 Announce Type: replace Abstract: Process reward models (PRMs) provide fine-grained supervision for reasoning, but reliable PRMs often require step annotations or heavy verification pipelines, making them costly to scale and refresh during online RL. Implicit PRMs reduce this cost by training log-likelihood-ratio rewards from trajectory-level outcome labels. However, the log-ratio is constra