Project benchmark

ARC-AGI-1 · Abstract reasoning

ARC Prize · 289 configurations with published scores. Review each metric and its test conditions separately.

Report identity & conditions

Data as of2026-11-03· Report published—. See individual records for run dates and test conditions.

Leaderboard updated

  • Run configurations remain separate, including different reasoning settings.
  • Individual test dates are not given. Update dates and model release dates from the report remain separate and do not substitute for actual test dates.
  • Costs follow the report’s run scope and dollar amounts; they are not model API unit prices.
  • ARC-AGI-1 · Semi-private evaluation set. Benchmark versions are compared separately.
  • Task score on the semi-private evaluation set, as a percentage. It is not ranked across benchmark versions or evaluation scopes.
  • Cost per task as reported, in USD/task. Total costs and task counts are not invented. This is neither cost per successful task nor a current API quote.
  • Human reference results are shown separately and excluded from model/system rankings and cost comparisons.
Read the original report ↗

Compare metrics within the same report. Each chart explains its direction and scope.
See the details for full values and run conditions.

Task score

Task score on the semi-private evaluation set, as a percentage. It is not ranked across benchmark versions or evaluation scopes.: showing 8 of 289 records with values. See details for configurations and dates.

Chart data · 8 entries
Name%
Claude Fable 5 (Max)98.5
Claude Fable 5 (XHigh)98.5
Claude Opus 5.5 (High)98.5
Gemini 3.8 Flash (High)98.5
Gemini 4 Argon (High)98.5
Gemini 4 Argon (Medium)98.5
GPT-6 Astra (High)98.5
GPT-6 Astra (XHigh)98.5

Cost per task

Cost per task as reported, in USD/task. Total costs and task counts are not invented. This is neither cost per successful task nor a current API quote.: showing 8 of 289 records with values. See details for configurations and dates.

Chart data · 8 entries
NameUSD / Task
GPT-5 Nano (Minimal)0.002
GPT-4.1-Nano0.002
GPT-6 Luna (None)0.002
DeepSeek V4 Flash 0731 (None)0.002
Qwen3-235b-a22b Instruct (25/07)0.003
GPT-6 Luna (Low)0.003
GPT-5 Nano (Low)0.003
Nemotron 3.5 Super VL (None)0.004

Human reference

Human baselines are shown separately and are not ranked alongside models and systems.

Stem Grad

Task score
98%
Cost per task
10 USD / Task

Human Panel

Task score
98%
Cost per task
17 USD / Task

Full results & model versions

289 / 289configurations have published scores in this scope. Scored configurations are listed; missing values remain “Not provided.” Different run configurations of the same model are listed separately.

Gemini 2.5 Pro (Thinking 1K)

Model reported by the benchmark

As reportedOrganization: Google

gemini-2-5-pro-2025-06-17-thinking · CoT

Release time reported by the benchmark:

Task score
16%
Cost per task
0.0573 USD / Task

Gemini 3.1 Pro (Preview)

Model reported by the benchmark

As reportedOrganization: Google

CoT

Release time reported by the benchmark:

Task score
98%
Cost per task
0.5223 USD / Task

ARChitects

Model reported by the benchmark

As reportedOrganization: ARC Prize 2024

Kaggle · Custom

Release time reported by the benchmark:

Task score
56%
Cost per task
0.2 USD / Task

GPT-5.6 Luna 2026-07-30 (None)

Model reported by the benchmark

As reportedOrganization: OpenAI

openai-gpt-5-6-luna-2026-07-30 · CoT

Release time reported by the benchmark:

Task score
3.333333333333333%
Cost per task
0.004658 USD / Task

GPT-5.4 Mini (XHigh)

Model reported by the benchmark

As reportedOrganization: OpenAI

gpt-5-4-mini-reasoning · CoT

Release time reported by the benchmark:

Task score
63.67%
Cost per task
0.51 USD / Task

Gemini 3.8 Flash (High)

Model reported by the benchmark

As reportedOrganization: Google

google-gemini-3-8-flash · CoT

Release time reported by the benchmark:

Task score
98.5%
Cost per task
0.205707 USD / Task

DeepSeek V4 Pro 0813 (High)

Model reported by the benchmark

As reportedOrganization: DeepSeek

deepseek-v4-pro-0813 · CoT

Release time reported by the benchmark:

Task score
87.16666666666666%
Cost per task
0.307073 USD / Task

Claude Opus 5 (Max)

Model reported by the benchmark

As reportedOrganization: Anthropic

anthropic-claude-opus-5 · CoT

Release time reported by the benchmark:

Task score
97.5%
Cost per task
0.7 USD / Task

Claude Fable 5 (Max)

Model reported by the benchmark

As reportedOrganization: Anthropic

anthropic-claude-fable-5 · CoT

Release time reported by the benchmark:

Task score
98.5%
Cost per task
2.110158 USD / Task

GPT-5.6 Terra (Medium)

Model reported by the benchmark

As reportedOrganization: OpenAI

openai-gpt-5-6-terra · CoT

Release time reported by the benchmark:

Task score
77%
Cost per task
0.13 USD / Task

GPT-5.1 (Thinking, Medium)

Model reported by the benchmark

As reportedOrganization: OpenAI

gpt-5-1-reasoning · CoT

Release time reported by the benchmark:

Task score
57.67%
Cost per task
0.3149 USD / Task

Claude Opus 4.8 (Max)

Model reported by the benchmark

As reportedOrganization: Anthropic

anthropic-opus-4-8 · CoT

Release time reported by the benchmark:

Task score
92.5%
Cost per task
2.332 USD / Task

Claude Opus 4 (Thinking 1K)

Model reported by the benchmark

As reportedOrganization: Anthropic

claude-opus-4-20250514-thinking · CoT

Release time reported by the benchmark:

Task score
27%
Cost per task
0.5021 USD / Task

GPT-5.2 Pro (High)

Model reported by the benchmark

As reportedOrganization: OpenAI

gpt-5-2-pro-2025-12-11-thinking · CoT

Release time reported by the benchmark:

Task score
85.67%
Cost per task
5.8694 USD / Task

DeepSeek V4.1 Flash (Low)

Model reported by the benchmark

As reportedOrganization: DeepSeek

deepseek-v4-1-flash · CoT

Release time reported by the benchmark:

Task score
90.5%
Cost per task
0.047985 USD / Task

GLM-5.3-Flash (Low)

Model reported by the benchmark

As reportedOrganization: Z.ai

zai-glm-5-3-flash · CoT

Release time reported by the benchmark:

Task score
46.99999999999999%
Cost per task
0.020154 USD / Task

GPT-5.1 (Thinking, High)

Model reported by the benchmark

As reportedOrganization: OpenAI

gpt-5-1-reasoning · CoT

Release time reported by the benchmark:

Task score
72.83%
Cost per task
0.674 USD / Task

DeepSeek V4.1 Flash 1017 (Max)

Model reported by the benchmark

As reportedOrganization: DeepSeek

deepseek-v4-1-flash-1017 · CoT

Release time reported by the benchmark:

Task score
93.5%
Cost per task
0.064591 USD / Task

Claude Fable 5 (Low)

Model reported by the benchmark

As reportedOrganization: Anthropic

anthropic-claude-fable-5 · CoT

Release time reported by the benchmark:

Task score
90.5%
Cost per task
0.50769 USD / Task

GPT-6 Sol (XHigh)

Model reported by the benchmark

As reportedOrganization: OpenAI

openai-gpt-6-sol · CoT

Release time reported by the benchmark:

Task score
92.66666666666664%
Cost per task
0.102302 USD / Task