Bilevel Optimization of Synthetic Trajectories for Multi-Turn LLM Fine-Tuning 事件

Bilevel Optimization of Synthetic Trajectories for Multi-Turn LLM Fine-Tuning arXiv:2605.24743v1 Announce Type: cross Abstract: While LLMs excel at single-turn generation, they struggle with long-horizon, multi-turn interactions. Offline reinforcement learning (RL) offers a scalable approach, yet its performance hinges on the availability and quality of multi-turn trajectory data. A common remedy is to augment training with synthetic trajectories generated by LLMs or simulators, but synthetic d