Counterfactual Evaluation Reveals Hidden Capability Profiles in Clinical LLMs and Agents 事件

Counterfactual Evaluation Reveals Hidden Capability Profiles in Clinical LLMs and Agents arXiv:2605.30590v1 Announce Type: cross Abstract: Two clinical AI systems can score nearly identically on coverage-based rubrics yet behave radically differently when their patient inputs change: one updates its recommendations to match the new clinical signal, while the other produces the same output regardless. We introduce the Causal Sensitivity Score (CSS), a pre-registered interventional metric that mu