CRAB-Bench: Evaluating LLM Agents under Complex Task Dependencies and Human-aligned User Simulation 事件

CRAB-Bench: Evaluating LLM Agents under Complex Task Dependencies and Human-aligned User Simulation arXiv:2606.01815v1 Announce Type: new Abstract: Evaluating LLM agents in realistic service scenarios requires complex task dependencies, imperfect user behavior, and an evaluation that accommodates multiple valid solutions. We introduce CRAB-Bench (Constraint-based Realistic Agent Benchmark) and RUSE (Realistic User Simulation Engine) to address this gap. CRAB-Bench generates tasks via a constrai