SWE-Bench Pro

Dataset Summary SWE-Bench Pro is a challenging, enterprise-level dataset for testing agent ability on long-horizon software engineering tasks. Paper: https://static.scale.com/uploads/654197dc94d34f66c0f5184e/SWEAP_Eval_Scale%20(9).pdf See the related evaluation Github: https://github.com/scaleapi/SWE-bench_Pro-os Datas

Papers using SWE-Bench Pro (3)

Terminus-4B: Can a Smaller Model Replace Frontier LLMs at Agentic Execution Tasks?2026CodeScout: An Effective Recipe for Reinforcement Learning of Code Search Agents2026Toward Training Superintelligent Software Agents through Self-Play SWE-RL2025