When LLMs Stop Following Steps: A Diagnostic Study of Procedural Execution in Language Models 事件

When LLMs Stop Following Steps: A Diagnostic Study of Procedural Execution in Language Models arXiv:2605.00817v3 Announce Type: replace Abstract: Large language models (LLMs) often achieve strong performance on reasoning benchmarks, but final-answer accuracy alone does not show whether they faithfully execute the procedure specified in a prompt. We introduce a controlled diagnostic benchmark for procedural execution, where models are given a step-wise arithmetic procedure and two numeric inputs