SIFT: Selective-Index For Fast Compute of RAG Prefill by Exploiting Attention Invariance 事件

SIFT: Selective-Index For Fast Compute of RAG Prefill by Exploiting Attention Invariance arXiv:2606.09441v1 Announce Type: new Abstract: Retrieval-Augmented Generation (RAG) injects LLM queries with relevant documents to improve response quality. This injection increases prompt length and slows time to first token (TTFT). Unlike standard queries, RAG queries have a unique property of context reuse where the same documents recur across user queries. Thus, fully recomputing documents for every RA