Beware of the Batch Size: Hyperparameter Bias in Evaluating LoRA 事件

Beware of the Batch Size: Hyperparameter Bias in Evaluating LoRA arXiv:2602.09492v2 Announce Type: replace-cross Abstract: Low-rank adaptation (LoRA) is a standard approach for fine-tuning large language models, yet its many variants report conflicting empirical gains, often on the same benchmarks. We show that these contradictions arise from a single overlooked factor: the batch size. When properly tuned, vanilla LoRA often matches the performance of more complex variants. We further propose a