Project benchmark

Aider Polyglot code-editing snapshot

Aider Benchmarks · 69 configurations with published scores. Review each metric and its test conditions separately.

Report identity & conditions

Data as of2026-10-14· Report published—. See individual records for run dates and test conditions.

The benchmark has 225 exercises across C++, Go, Java, JavaScript, Python and Rust. Some runs report 223 or 224 test cases; the actual count is retained for each run.
The observation date is when we retrieved the data. Run dates, samples, reasoning settings, edit formats, tool versions and commits are recorded separately. This leaderboard includes historical runs and is not a current overall model ranking.
Display names and CLI aliases do not establish an exact snapshot, so they are not automatically linked to catalog versions. Dollar costs for a run are not reusable API unit prices, and runs with different settings are not controlled comparisons.

Read the original report ↗

Test sampleLeft metricRight metric

Compare metrics within the same report. Each chart explains its direction and scope.
See the details for full values and run conditions.

Percent correct

Results from the original runs, rather than a general capability score. Dates, reasoning settings, edit formats and samples remain separate.: showing 8 of 63 records with values. See details for configurations and dates.

Chart data · 8 entries

Name	%
gpt-5 (high)	88
gpt-5 (medium)	86.7
o3-pro (high)	84.9
gemini-2.5-pro-preview-06-05 (32k think)	83.1
gpt-5 (low)	81.3
o3 (high)	81.3
grok-4 (high)	79.6
gemini-2.5-pro-preview-06-05 (default think)	79.1

Correct edit format

Valid edit format rate. This is separate from task completion accuracy and is not combined into an overall score.: showing 8 of 63 records with values. See details for configurations and dates.

Chart data · 8 entries

Name	%
Codestral 25.01	100
DeepSeek R1 + claude-3-5-sonnet-20241022	100
Gemini 2.0 Pro exp-02-05	100
gemini-2.0-flash-exp	100
gemini-2.5-pro-preview-06-05 (default think)	100
gemma-3-27b-it	100
gpt-4o-mini-2024-07-18	100
Grok 3 Mini Beta (low)	100

Full results & model versions

69 / 69configurations have published scores in this scope. Scored configurations are listed; missing values remain “Not provided.” Different run configurations of the same model are listed separately.

gpt-5 (high)

Model reported by the benchmark

Run2025-08-23 · 225problems · Formatdiff· Reasoning levelhigh
Tools0.86.2.dev· Submission32faf82

Percent correct: 88%
Correct edit format: 91.6%

Run command

aider --model openai/gpt-5

gpt-5 (medium)

Model reported by the benchmark

Run2025-08-25 · 225problems · Formatdiff· Reasoning levelmedium
Tools0.86.2.dev· Submission32faf82

Percent correct: 86.7%
Correct edit format: 88.4%

Run command

aider --model openai/gpt-5

o3-pro (high)

Model reported by the benchmark

Run2025-06-28 · 225problems · Formatdiff· Reasoning levelhigh
Tools0.85.1.dev· Submission5318380

Percent correct: 84.9%
Correct edit format: 97.8%

Run command

aider --model o3-pro

gemini-2.5-pro-preview-06-05 (32k think)

Model reported by the benchmark

Run2025-06-06 · 225problems · Formatdiff-fenced· Reasoning levelNot listed
Tools0.84.1.dev· Submissionf827f22

Percent correct: 83.1%
Correct edit format: 99.6%

Run command

aider --model gemini/gemini-2.5-pro-preview-06-05 --thinking-tokens 32k

gpt-5 (low)

Model reported by the benchmark

Run2025-08-25 · 225problems · Formatdiff· Reasoning levellow
Tools0.86.2.dev· Submission32faf82

Percent correct: 81.3%
Correct edit format: 86.7%

Run command

aider --model openai/gpt-5

o3 (high)

Model reported by the benchmark

Run2025-06-25 · 225problems · Formatdiff· Reasoning levelhigh
Tools0.84.1.dev· Submissionc48fea6

Percent correct: 81.3%
Correct edit format: 94.7%

Run command

aider --model o3 --reasoning-effort high

grok-4 (high)

Model reported by the benchmark

Run2025-07-11 · 225problems · Formatdiff· Reasoning levelhigh
Tools0.85.2.dev· Submissionf7870b6-dirty

Percent correct: 79.6%
Correct edit format: 97.3%

Run command

aider --model openrouter/x-ai/grok-4

gemini-2.5-pro-preview-06-05 (default think)

Model reported by the benchmark

Run2025-06-06 · 225problems · Formatdiff-fenced· Reasoning levelNot listed
Tools0.84.1.dev· Submission4c161f9-dirty

Percent correct: 79.1%
Correct edit format: 100%

Run command

aider --model gemini/gemini-2.5-pro-preview-06-05

o3 (high) + gpt-4.1

Model reported by the benchmark

Run2025-06-27 · 224problems · Formatarchitect· Reasoning levelhigh
Tools0.85.1.dev· Submission4f4f00f-dirty

Percent correct: 78.2%
Correct edit format: 100%

Run command

aider --model o3

o3

Model reported by the benchmark

Run2025-06-25 · 225problems · Formatdiff· Reasoning levelNot listed
Tools0.84.1.dev· Submissionc48fea6

Percent correct: 76.9%
Correct edit format: 93.8%

Run command

aider --model o3

Gemini 2.5 Pro Preview 05-06

Model reported by the benchmark

Run2025-05-07 · 225problems · Formatdiff-fenced· Reasoning levelNot listed
Tools0.82.4.dev· Submission3b08327-dirty

Percent correct: 76.9%
Correct edit format: 97.3%

Run command

aider --model gemini/gemini-2.5-pro-preview-05-06

DeepSeek-V3.2-Exp (Reasoner)

Model reported by the benchmark

Run2025-10-03 · 225problems · Formatdiff· Reasoning levelNot listed
Tools0.86.2.dev· Submissioncbb5376

Percent correct: 74.2%
Correct edit format: 97.3%

Run command

aider --model deepseek/deepseek-reasoner

Gemini 2.5 Pro Preview 03-25

Model reported by the benchmark

Run2025-04-12 · 225problems · Formatdiff-fenced· Reasoning levelNot listed
Tools0.81.3.dev· Submission0282574

Percent correct: 72.9%
Correct edit format: 92.4%

Run command

aider --model gemini/gemini-2.5-pro-preview-03-25

claude-opus-4-20250514 (32k thinking)

Model reported by the benchmark

Run2025-05-25 · 225problems · Formatdiff· Reasoning levelNot listed
Tools0.83.3.dev· Submission9ef3211

Percent correct: 72%
Correct edit format: 97.3%

Run command

aider --model claude-opus-4-20250514

o4-mini (high)

Model reported by the benchmark

Run2025-04-16 · 225problems · Formatdiff· Reasoning levelNot listed
Tools0.82.1.dev· Submissionb66901f-dirty

Percent correct: 72%
Correct edit format: 90.7%

Run command

aider --model o4-mini

DeepSeek R1 (0528)

Model reported by the benchmark

Run2025-06-06 · 224problems · Formatdiff· Reasoning levelNot listed
Tools0.84.1.dev· Submission4c161f9-dirty

Percent correct: 71.4%
Correct edit format: 94.6%

Run command

aider --model deepseek/deepseek-reasoner

claude-opus-4-20250514 (no think)

Model reported by the benchmark

Run2025-05-25 · 225problems · Formatdiff· Reasoning levelNot listed
Tools0.83.3.dev· Submission9ef3211

Percent correct: 70.7%
Correct edit format: 98.7%

Run command

aider --model claude-opus-4-20250514

DeepSeek-V3.2-Exp (Chat)

Model reported by the benchmark

Run2025-10-03 · 225problems · Formatdiff· Reasoning levelNot listed
Tools0.86.2.dev· Submissioncbb5376

Percent correct: 70.2%
Correct edit format: 98.2%

Run command

aider --model deepseek/deepseek-chat

claude-3-7-sonnet-20250219 (32k thinking tokens)

Model reported by the benchmark

Run2025-02-24 · 225problems · Formatdiff· Reasoning levelNot listed
Tools0.75.1.dev· Submission60d11a6, 93edbda

Percent correct: 64.9%
Correct edit format: 97.8%

Run command

aider --model anthropic/claude-3-7-sonnet-20250219 --thinking-tokens 32k

DeepSeek R1 + claude-3-5-sonnet-20241022

Model reported by the benchmark

Run2025-01-23 · 225problems · Formatarchitect· Reasoning levelNot listed
Tools0.72.3.dev· Submission05a77c7

Percent correct: 64%
Correct edit format: 100%

Run command

aider --architect --model r1 --editor-model sonnet