← all leaderboards

SWE-bench Verified swe-bench-verified Leaderboard

SWE-bench Verified — 500 human-validated GitHub issues from real Python repos that an agent must resolve by producing a patch that passes the hidden test suite. The most-cited frontier benchmark for autonomous software engineering. Score is the % of issues resolved. · Metric: % Resolved (higher is better)

Source ↗

#Model% ResolvedPaper
1live-SWE-agent + Claude 4.5 Opus medium (20251101)79.20link
2Sonar Foundation Agent + Claude 4.5 Opus79.20link
3TRAE + Doubao-Seed-Code78.80link
4live-SWE-agent + Gemini 3 Pro Preview (2025-11-18)77.40link
5Atlassian Rovo Dev (2025-09-02)76.80link
6EPAM AI/Run Developer Agent v20250719 + Claude 4 Sonnet76.80link
7mini-SWE-agent + Claude 4.5 Opus (high reasoning)76.80link
8ACoder76.40link
9mini-SWE-agent + Gemini 3 Flash (high reasoning)75.80link
10mini-SWE-agent + MiniMax M2.5 (high reasoning)75.80link
11mini-SWE-agent + Claude Opus 4.675.60link
12Warp75.60link
13TRAE + Claude Sonnet 4 + Opus 4 + Sonnet 3.7 + Gemini 2.5 Pro75.20link
14Harness AI74.80link
15Sonar Foundation Agent + Claude 4.5 Sonnet74.80link
16JoyCode + Claude 4 Sonnet + GPT-4.174.60link
17Lingxi-v1.5_claude-4-sonnet-2025051474.60link
18mini-SWE-agent + Claude 4.5 Opus medium (20251101)74.40link
19Prometheus-v1.2.1 + GPT-574.40link
20Refact.ai Agent + Claude 4 Sonnet + o4-mini74.40link
21mini-SWE-agent + Gemini 3 Pro Preview (2025-11-18)74.20link
22Salesforce AI Research SAGE (OpenHands)73.80link
23Tools + Claude 4 Opus (2025-05-22)73.20link
24Salesforce AI Research SAGE (bash-only)73.00link
25mini-SWE-agent + GLM-5 (high reasoning)72.80link
26mini-SWE-agent + GPT-5-2 Codex72.80link
27mini-SWE-agent + GPT-5-2 (high reasoning)72.80link
28Tools + Claude 4 Sonnet (2025-05-22)72.40link
29mini-SWE-agent + GPT-5.2 (2025-12-11) (high reasoning)71.80link
30OpenHands + GPT-571.80link
SWE-bench Verified swe-bench-verified Leaderboard