SWE-bench Verified

Dataset Summary SWE-bench Verified is a subset of 500 samples from the SWE-bench test set, which have been human-validated for quality. SWE-bench is a dataset that tests systems’ ability to solve GitHub issues automatically. See this post for more details on the human-validation process. The dataset collects 500 test I

Papers using SWE-bench Verified (14)

Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning2026Training Long-Context, Multi-Turn Software Engineering Agents with Reinforcement Learning2025 · 21 citesStabilizing Long-term Multi-turn Reinforcement Learning with Gated Rewards2025 · 2 citesSWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution2025 · 2 citesRollout Pass-Rate Control: Steering Binary-Reward RL Toward Its Most Informative Regime2026Revisiting DAgger in the Era of LLM-Agents2026CODESKILL: Learning Self-Evolving Skills for Coding Agents2026RTMC: Step-Level Credit Assignment via Rollout Trees2026CodeScout: An Effective Recipe for Reinforcement Learning of Code Search Agents2026KAT-Coder-V2 Technical Report2026Toward Training Superintelligent Software Agents through Self-Play SWE-RL2025Reinforcement Learning-Guided Chain-of-Draft for Token-Efficient Code Generation2025Kimi K2: Open Agentic Intelligence2025Satori-SWE: Evolutionary Test-Time Scaling for Sample-Efficient Software Engineering2025