From "Weak" Signals to Strong Models: Preference Delta Aggregation with LoRA Merging 事件

From "Weak" Signals to Strong Models: Preference Delta Aggregation with LoRA Merging arXiv:2606.00357v1 Announce Type: new Abstract: Training strong large language models (LLMs) requires high-quality supervision, which is often scarce. Recent work shows that paired preference data from weak-weaker model pairs (e.g., Qwen3 4B over 1.7B), despite the limited quality of individual responses, can provide an effective supervision signal through relative quality deltas, which we term a "weak" signal.