Recovering Diversity Without Losing Alignment: A DPO Recipe for Post-Trained LLMs 事件

Recovering Diversity Without Losing Alignment: A DPO Recipe for Post-Trained LLMs arXiv:2605.30021v1 Announce Type: new Abstract: Many open-ended instructions have multiple valid answers that users can benefit from seeing, but post-training often narrows an LLM's output space toward a small set of canonical responses. We introduce REDIPO, an offline DPO data-construction pipeline for recovering distinct valid answer modes while preserving the alignment benefits of the instruct model. For each p