LiveCodeBench code generation · 2024-08-01 to 2025-05-01

Report identity & conditions

Data as of2026-10-19· Report published—. See individual records for run dates and test conditions.

• Code generation is compared within problem-release windows. Overall, easy, medium and hard scores use the arithmetic mean of per-problem Pass@1, shown to one decimal place.
• Per-problem Pass@1 is already a 0–100 percentage and is not multiplied by 100 again. Question counts are shown separately; a difficulty with no problems remains missing.
• Problems come from LeetCode, AtCoder and Codeforces. Release windows and question counts are neither model run dates nor counts of independent generation samples.
• Estimated training cutoffs are claims by the benchmark project, rather than model release dates. Configurations with cutoffs on or after the window start retain their results and a separate flag, but are excluded from ranking charts.
• Reasoning settings and call templates remain separate for models with the same name. Generation sampling, run dates and test settings follow the relevant report.
• This report compares code generation within the stated problem window. Model selection conclusions apply to that scope; other windows and evaluation settings are compared separately.
• The official default problem window is 2024-08-01 to 2025-05-01, inclusive, with 454 problems in this group.