Reproducing, Analyzing, and Detecting Reward Hacking in Rubric-Based Reinforcement Learning 事件

Reproducing, Analyzing, and Detecting Reward Hacking in Rubric-Based Reinforcement Learning arXiv:2606.04923v1 Announce Type: cross Abstract: Rubric-based reinforcement learning (RL) uses an LLM-as-a-Judge (LaaJ) to score model outputs according to rubrics as rewards. However, policy models may exploit latent biases in the judge, leading to reward hacking and ineffective or unsafe training outcomes. In real-world rubric-based RL, such hacking behaviors are often subtle and entangled with multip