Cartridges at Scale: Training Modular KV Caches over Large Document Collections 事件

Cartridges at Scale: Training Modular KV Caches over Large Document Collections arXiv:2606.04557v1 Announce Type: new Abstract: Large Language Models can reason over long contexts, yet prefilling millions of tokens is wasteful as much of the content remains static across queries. Cartridges address this by distilling document collections into reusable key-value (KV) caches that eliminate prefilling while preserving accuracy. A critical limitation of this approach is that cartridges are monolith