Project benchmark

LiveCodeBench code generation · 2024-08-01 to 2025-05-01

LiveCodeBench · 28 configurations with published scores. Review each metric and its test conditions separately.

Report identity & conditions

Data as of2026-10-19· Report published—. See individual records for run dates and test conditions.

Code generation is compared within problem-release windows. Overall, easy, medium and hard scores use the arithmetic mean of per-problem Pass@1, shown to one decimal place.
Per-problem Pass@1 is already a 0–100 percentage and is not multiplied by 100 again. Question counts are shown separately; a difficulty with no problems remains missing.
Problems come from LeetCode, AtCoder and Codeforces. Release windows and question counts are neither model run dates nor counts of independent generation samples.
Estimated training cutoffs are claims by the benchmark project, rather than model release dates. Configurations with cutoffs on or after the window start retain their results and a separate flag, but are excluded from ranking charts.
Reasoning settings and call templates remain separate for models with the same name. Generation sampling, run dates and test settings follow the relevant report.
This report compares code generation within the stated problem window. Model selection conclusions apply to that scope; other windows and evaluation settings are compared separately.
The official default problem window is 2024-08-01 to 2025-05-01, inclusive, with 454 problems in this group.

Left metricRight metric

Compare metrics within the same report. Each chart explains its direction and scope.
See the details for full values and run conditions.

Pass@1

Mean per-problem Pass@1 across 454 problems released from 2024-08-01 to 2025-05-01, inclusive, retaining the official one-decimal precision. Problem dates are not model run dates.: showing 8 of 28 records with values. See details for configurations and dates.

Chart data · 8 entries

Name	%
O4-Mini (High)	80.2
O3 (High)	75.8
O4-Mini (Medium)	74.2
Gemini-2.5-Pro-06-05	73.6
DeepSeek-R1-0528	73.1
Gemini-2.5-Pro-05-06	71.8
EXAONE-4.0-32B	70
OpenReasoning-Nemotron-32B	69.8

Easy

Mean per-problem Pass@1 across 110 problems released from 2024-08-01 to 2025-05-01, inclusive, retaining the official one-decimal precision. Problem dates are not model run dates.: showing 8 of 28 records with values. See details for configurations and dates.

Chart data · 8 entries

Name	%
Gemini-2.5-Flash-04-17	99.1
Gemini-2.5-Flash-05-20	99.1
Gemini-2.5-Pro-06-05	99.1
O3 (High)	99.1
O3-Mini-2025-01-31 (High)	99.1
O3-Mini-2025-01-31 (Low)	99.1
O3-Mini-2025-01-31 (Med)	99.1
O4-Mini (High)	99.1

Full results & model versions

28 / 28configurations have published scores in this scope. Scored configurations are listed; missing values remain “Not provided.” Different run configurations of the same model are listed separately.

O4-Mini (High)

Model reported by the benchmark

As reported · Model details ↗

Problem publication window:2024-08-01to2025-05-01(inclusive) ·454problems

Estimated training cutoff:2023-04-30

Evaluation prompt template:OpenAIReason

Pass@1: 80.2%
Easy: 99.1%
Medium: 89.4%
Hard: 63.5%

Gemini-2.5-Pro-05-06

Model reported by the benchmark

As reported · Model details ↗

Problem publication window:2024-08-01to2025-05-01(inclusive) ·454problems

Estimated training cutoff:2023-04-30

Evaluation prompt template:GeminiThinking

Pass@1: 71.8%
Easy: 98.2%
Medium: 82.3%
Hard: 50.2%

OpenReasoning-Nemotron-32B

Model reported by the benchmark

As reported · Model details ↗

Problem publication window:2024-08-01to2025-05-01(inclusive) ·454problems

Estimated training cutoff:2024-04-01

Evaluation prompt template:DeepSeekR1

Pass@1: 69.8%
Easy: 98.3%
Medium: 81.4%
Hard: 46.3%

EXAONE-4.0-32B

Model reported by the benchmark

As reported · Model details ↗

Problem publication window:2024-08-01to2025-05-01(inclusive) ·454problems

Estimated training cutoff:2024-04-01

Evaluation prompt template:EXAONE

Pass@1: 70%
Easy: 98.4%
Medium: 82.3%
Hard: 46.2%

GPT-4O-2024-08-06

Model reported by the benchmark

As reported · Model details ↗

Problem publication window:2024-08-01to2025-05-01(inclusive) ·454problems

Estimated training cutoff:2023-04-30

Evaluation prompt template:OpenAIChat

Pass@1: 29.5%
Easy: 82.5%
Medium: 26%
Hard: 3.3%

DeepSeek-V3

Model reported by the benchmark

As reported · Model details ↗

Problem publication window:2024-08-01to2025-05-01(inclusive) ·454problems

Estimated training cutoff:2024-06-30

Evaluation prompt template:DeepSeekAPI

Pass@1: 27.2%
Easy: 64.3%
Medium: 27.9%
Hard: 6.7%

GPT-4O-mini-2024-07-18

Model reported by the benchmark

As reported · Model details ↗

Problem publication window:2024-08-01to2025-05-01(inclusive) ·454problems

Estimated training cutoff:2023-04-30

Evaluation prompt template:OpenAIChat

Pass@1: 27.5%
Easy: 81.9%
Medium: 18.9%
Hard: 3.9%

Claude-Opus-4 (Thinking)

Model reported by the benchmark

As reported · Model details ↗

Problem publication window:2024-08-01to2025-05-01(inclusive) ·454problems

Estimated training cutoff:2023-04-30

Evaluation prompt template:Claude3Thinking

Pass@1: 56.6%
Easy: 98.2%
Medium: 70.9%
Hard: 24.1%