SoLoPO: Unlocking Long-Context Capabilities in LLMs via Short-to-Long Preference Optimization 事件

SoLoPO: Unlocking Long-Context Capabilities in LLMs via Short-to-Long Preference Optimization arXiv:2505.11166v3 Announce Type: replace Abstract: Despite advances in pretraining with extended context sizes, large language models (LLMs) still face challenges in effectively utilizing real-world long-context information, primarily due to insufficient long-context alignment caused by data quality issues, training inefficiencies, and the lack of well-designed optimization objectives. To address thes