AGENTSERVESIM: A Hardware-aware Simulator for Multi-Turn LLM Agent Serving 事件

AGENTSERVESIM: A Hardware-aware Simulator for Multi-Turn LLM Agent Serving arXiv:2606.09613v1 Announce Type: cross Abstract: Multi-turn LLM agents interleave model calls with external tool invocations, shifting serving from stateless request processing to stateful program execution. Serving these workloads requires scheduling, KV-cache management, and routing policies that use program-level context, including turn dependencies, tool-induced gaps, and reusable KV state. Evaluating such policies