SWE-Bench Pro

Dataset Summary SWE-Bench Pro is a challenging, enterprise-level dataset for testing agent ability on long-horizon software engineering tasks. Paper: https://static.scale.com/uploads/654197dc94d34f66c0f5184e/SWEAP_Eval_Scale%20(9).pdf See the related evaluation Github: https://github.com/scaleapi/SWE-bench_Pro-os Datas

Papers using SWE-Bench Pro (5)

Sakana Fugu Technical Report2026FastContext: Training Efficient Repository Explorer for Coding Agents2026SkillsVote: Lifecycle Governance of Agent Skills from Collection, Recommendation to Evolution2026Retrospective Harness Optimization: Improving LLM Agents via Self-Preference over Trajectory Rollouts2026Socratic-SWE: Self-Evolving Coding Agents via Trace-Derived Agent Skills2026