Directional Alignment Mitigates Reward Hacking in Reinforcement Learning for Language Models 事件

Directional Alignment Mitigates Reward Hacking in Reinforcement Learning for Language Models arXiv:2605.25189v1 Announce Type: cross Abstract: Reward hacking arises when a model improves a proxy reward by exploiting shortcuts rather than solving the intended task. We study this failure mode through the geometry of reinforcement learning updates in language models and argue that hacking emerges when optimization drifts away from a stable low-dimensional learning trajectory. We analyze this drift