PBT-Bench: Benchmarking AI Agents on Property-Based Testing 事件

PBT-Bench: Benchmarking AI Agents on Property-Based Testing arXiv:2605.15229v3 Announce Type: replace-cross Abstract: Existing code benchmarks measure whether an agent can produce any test that reproduces a known bug, or whether it can produce a patch that fixes a described issue. Neither isolates the distinct skill of property-based testing: deriving a semantic invariant from documentation, and then constructing an input-generation strategy precise enough to make a random search reveal the