Efficient Preference Poisoning Attack on Offline RLHF 事件

Efficient Preference Poisoning Attack on Offline RLHF arXiv:2605.02495v2 Announce Type: replace-cross Abstract: Offline Reinforcement Learning from Human Feedback (RLHF) pipelines such as Direct Preference Optimization (DPO) train on a pre-collected preference dataset, which makes them vulnerable to preference poisoning attack. We study label flip attacks against log-linear DPO. We first illustrate that flipping one preference label induces a parameter-independent shift in the DPO gradient. Usi