Fine-Tuning Without Forgetting In-Context Learning: A Theoretical Analysis of Linear Attention Models 事件

Fine-Tuning Without Forgetting In-Context Learning: A Theoretical Analysis of Linear Attention Models arXiv:2602.23197v2 Announce Type: replace Abstract: Transformer-based large language models exhibit in-context learning, enabling adaptation to downstream tasks via few-shot prompting with demonstrations. In practice, such models are often fine-tuned to improve zero-shot performance on downstream tasks, allowing them to solve tasks without examples and thereby reducing inference costs. However,