OP-LoRA: The Blessing of Dimensionality 事件

OP-LoRA: The Blessing of Dimensionality arXiv:2412.10362v2 Announce Type: replace-cross Abstract: Low-rank adapters (LoRA) enable finetuning of large models with only a small number of parameters. However, they often suffer from an ill-conditioned loss landscape, leading to difficult optimization. Prior work addresses these challenges by aligning adapter updates with full finetuning gradients via custom optimizers, but these methods lack the flexibility to accommodate new adapter architectures