Finer Parameter Steps for Low-Rank PEFT: A Controlled Study with CP Tensor Adapters 事件

Finer Parameter Steps for Low-Rank PEFT: A Controlled Study with CP Tensor Adapters arXiv:2606.00428v1 Announce Type: cross Abstract: Low-rank adapters are usually compared by sweeping a small set of ranks, but the rank also fixes the resolution of the parameter budget. For a $2048{\times}2048$ OPT attention projection, increasing LoRA by one rank stores $4096$ trainable scalars, leaving large gaps between feasible low-budget adapter sizes. This paper asks whether a tensorized adapter with fine