NetKV: Network-Aware Decode Instance Selection for Disaggregated LLM Inference 事件

NetKV: Network-Aware Decode Instance Selection for Disaggregated LLM Inference arXiv:2606.03910v1 Announce Type: cross Abstract: Disaggregated LLM inference forces the KV cache to traverse the datacenter network before decoding begins, so transfer time enters directly into the Time to First Token (TTFT) budget. Current schedulers route on compute load and prefix-cache locality alone, ignoring the topological distance and dynamic congestion between prefill and decode instances. We close this gap