Searching the Internet for Challenging Benchmarks at Scale 事件

Searching the Internet for Challenging Benchmarks at Scale arXiv:2509.26619v3 Announce Type: replace Abstract: Many static benchmarks are beginning to saturate: as models rapidly improve, they achieve near-perfect scores on fixed test sets, leaving little headroom to expose genuine model weaknesses -- and even expert-curated challenge sets quickly saturate after hillclimbing. We present a fully automatic framework that searches the Internet at scale to construct challenging benchmarks without h