Reward Learning from Best-of-$N$ Preference Data: Targets, Tradeoffs, and Design Principles 事件

Reward Learning from Best-of-$N$ Preference Data: Targets, Tradeoffs, and Design Principles arXiv:2605.30619v1 Announce Type: cross Abstract: Best-of-$N$ sampling is widely used to construct pairwise preference data: $N$ candidates are drawn from a base distribution, and the best is paired with a rejected response. Despite its widespread use, what Bradley--Terry (BT) reward learning extracts from such data, and how to choose $N$ and the base distribution, remain unclear. We specialize a recent