SWE-Bench Pro

Dataset Summary SWE-Bench Pro is a challenging, enterprise-level dataset for testing agent ability on long-horizon software engineering tasks. Paper: https://static.scale.com/uploads/654197dc94d34f66c0f5184e/SWEAP_Eval_Scale%20(9).pdf See the related evaluation Github: https://github.com/scaleapi/SWE-bench_Pro-os Datas

Papers using SWE-Bench Pro (8)

Socratic-SWE: Self-Evolving Coding Agents via Trace-Derived Agent Skills2026Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference2026Exploration Structure in LLM Agents for Multi-File Change Localization2026Open-SWE-Traces: Advancing Dual-Mode Multilingual Distillation for Software Engineering Agents2026SkillsVote: Lifecycle Governance of Agent Skills from Collection, Recommendation to Evolution2026Evaluating Plan Compliance In Autonomous Programming Agents2026SWE-Adept: An LLM-Based Agentic Framework for Deep Codebase Analysis and Structured Issue Resolution2026Toward Training Superintelligent Software Agents through Self-Play SWE-RL2025