Unraveling LoRA Interference: Orthogonal Subspaces for Robust Model Merging 事件

Unraveling LoRA Interference: Orthogonal Subspaces for Robust Model Merging arXiv:2505.22934v2 Announce Type: replace Abstract: Fine-tuning large language models (LMs) for individual tasks yields strong performance but is expensive for deployment and storage. Recent works explore model merging to combine multiple task-specific models into a single multi-task model without additional training. However, existing merging methods often fail for models fine-tuned with low-rank adaptation (LoRA), due