Adaptive Preference Optimization with Uncertainty-aware Utility Anchor 事件

Adaptive Preference Optimization with Uncertainty-aware Utility Anchor arXiv:2509.10515v1 Announce Type: cross Abstract: Offline preference optimization methods are efficient for large language models (LLMs) alignment. Direct Preference optimization (DPO)-like learning, one of the most popular approaches, stands out for its efficiency in reward modeling. However, these methods typically follow the convention to use Bradley-Terry (BT) reward modeling that faces several critical assumptions, incl