Benchmark Leaderboards

Track which models hold the current SOTA on the benchmarks the field actually uses. Click a column header to sort; the leading row is highlighted.

15 tasks tracked

bigcodebench-complete

BigCodeBench Complete subset — code completion across 1,140 diverse function-level tasks from 139 libraries. Measures whether models can finish realistic code given partial context.

Metric: Pass@1 — higher is better · Source ↗

# ▲	Model / Paper	Pass@1	Δ vs SOTA	Links
1	Gemini-Exp-1206SOTA	62.4	—	search →
2	DeepSeek-V3	62.2	-0.20	search →
3	Llama-4-Maverick	61.4	-1.00	search →
4	GPT-4o-2024-05-13	61.1	-1.30	search →
5	Quasar-Alpha	60.6	-1.80	search →
6	Gemini-2.0-Flash-Exp	59.9	-2.50	search →
7	DeepSeek-Coder-V2-Instruct	59.7	-2.70	search →
8	DeepSeek-V2-Chat (2024-06-28)	59.4	-3.00	search →
9	Gemini-Exp-1114	59.3	-3.10	search →
10	GPT-4.1-Mini-2025-04-14	59.3	-3.10	search →
11	Claude-3.5-Haiku-20241022	59	-3.40	search →
12	GPT-4o-2024-11-20	58.9	-3.50	search →
13	Claude-3.5-Sonnet-20240620	58.6	-3.80	search →
14	GPT-4-Turbo-2024-04-09	58.2	-4.20	search →
15	Gemini-Exp-1121	58.1	-4.30	search →
16	Qwen2.5-Coder-32B-Instruct	58	-4.40	search →
17	Claude-3.5-Sonnet-20241022	57.5	-4.90	search →
18	Gemini-1.5-Pro-API-0514	57.5	-4.90	search →
19	Llama-3.3-70B-Instruct	57.5	-4.90	search →
20	Claude-3-Opus-20240229	57.4	-5.00	search →
21	GPT-4o-mini-2024-07-18	57.4	-5.00	search →
22	GPT-4-0613	57.2	-5.20	search →
23	Athene-V2-Chat	56.8	-5.60	search →
24	Qwen2.5-Coder-14B-Instruct	56.7	-5.70	search →
25	Athene-V2-Agent	56.1	-6.30	search →
26	Qwen2.5-72B-Instruct	55.9	-6.50	search →
27	Hermes-2-Theta-Llama-3-70B	55.6	-6.80	search →
28	Phi-4	55.4	-7.00	search →
29	Gemini-1.5-Flash-API-0514	55.1	-7.30	search →
30	DeepSeek-R1-Distill-Qwen-32B	54.9	-7.50	search →
31	FLARE: Fine-Grained Diagnostic Feedback for LLM Code Refinement	53.81	-8.59	arxiv.org →
32	Large Language Model Guided Self-Debugging Code Generation	41	-21.40	arxiv.org →

Benchmark Leaderboards

Loading leaderboards…