SWE-bench Pro

Dataset Summary SWE-Bench Pro is a challenging, enterprise-level dataset for testing agent ability on long-horizon software engineering tasks. Paper: https://static.scale.com/uploads/654197dc94d34f66c0f5184e/SWEAP_Eval_Scale%20(9).pdf See the related evaluation Github: https://github.com/scaleapi/SWE-bench_Pro-os Datas

Papers using SWE-bench Pro (12)

SEC-bench Pro: Can Language Models Solve Long-Horizon Software Security Tasks?2026Laguna M.1/XS.2 Technical Report2026Socratic-SWE: Self-Evolving Coding Agents via Trace-Derived Agent Skills2026SWE-Adept: An LLM-Based Agentic Framework for Deep Codebase Analysis and Structured Issue Resolution2026CodeScout: An Effective Recipe for Reinforcement Learning of Code Search Agents2026SWE-Replay: Efficient Test-Time Scaling for Software Engineering Agents2026Confucius Code Agent: Scalable Agent Scaffolding for Real-World Codebases2025Toward Training Superintelligent Software Agents through Self-Play SWE-RL2025The Dual-State Architecture for Reliable LLM Agents2025SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?2025Confucius Code Agent: An Open-sourced AI Software Engineer at Industrial Scale2025FastContext: Training Efficient Repository Explorer for Coding Agents2026