Mutation Testing of Task-Scoped State Oracles in Software-Agent Benchmarks: A Cross-Benchmark Empirical Study
Published in Research Square (preprint), 2026
A cross-benchmark empirical study that applies mutation testing to task-scoped state oracles, measuring how well benchmarks detect unintended side effects and calibrating oracle reliability across software-agent evaluation suites. Code is available in the side-effect-calibration-study repository.
Recommended citation: Sun, S. (2026). "Mutation Testing of Task-Scoped State Oracles in Software-Agent Benchmarks: A Cross-Benchmark Empirical Study." Research Square. doi:10.21203/rs.3.rs-10665114/v1
Download Paper