Project benchmark

ARC-AGI-1 · Abstract reasoning

ARC Prize · 289 configurations with published scores. Review each metric and its test conditions separately.

Report identity & conditions

Data as of2026-11-03· Report published—. See individual records for run dates and test conditions.

Leaderboard updated

  • Run configurations remain separate, including different reasoning settings.
  • Individual test dates are not given. Update dates and model release dates from the report remain separate and do not substitute for actual test dates.
  • Costs follow the report’s run scope and dollar amounts; they are not model API unit prices.
  • ARC-AGI-1 · Semi-private evaluation set. Benchmark versions are compared separately.
  • Task score on the semi-private evaluation set, as a percentage. It is not ranked across benchmark versions or evaluation scopes.
  • Cost per task as reported, in USD/task. Total costs and task counts are not invented. This is neither cost per successful task nor a current API quote.
  • Human reference results are shown separately and excluded from model/system rankings and cost comparisons.
Read the original report ↗

Compare metrics within the same report. Each chart explains its direction and scope.
See the details for full values and run conditions.

Task score

Task score on the semi-private evaluation set, as a percentage. It is not ranked across benchmark versions or evaluation scopes.: showing 8 of 289 records with values. See details for configurations and dates.

Chart data · 8 entries
Name%
Claude Fable 5 (Max)98.5
Claude Fable 5 (XHigh)98.5
Claude Opus 5.5 (High)98.5
Gemini 3.8 Flash (High)98.5
Gemini 4 Argon (High)98.5
Gemini 4 Argon (Medium)98.5
GPT-6 Astra (High)98.5
GPT-6 Astra (XHigh)98.5

Cost per task

Cost per task as reported, in USD/task. Total costs and task counts are not invented. This is neither cost per successful task nor a current API quote.: showing 8 of 289 records with values. See details for configurations and dates.

Chart data · 8 entries
NameUSD / Task
GPT-5 Nano (Minimal)0.002
GPT-4.1-Nano0.002
GPT-6 Luna (None)0.002
DeepSeek V4 Flash 0731 (None)0.002
Qwen3-235b-a22b Instruct (25/07)0.003
GPT-6 Luna (Low)0.003
GPT-5 Nano (Low)0.003
Nemotron 3.5 Super VL (None)0.004

Human reference

Human baselines are shown separately and are not ranked alongside models and systems.

Stem Grad

Task score
98%
Cost per task
10 USD / Task

Human Panel

Task score
98%
Cost per task
17 USD / Task

Full results & model versions

289 / 289configurations have published scores in this scope. Scored configurations are listed; missing values remain “Not provided.” Different run configurations of the same model are listed separately.

GPT-5.2 (High)

Model reported by the benchmark

As reportedOrganization: OpenAI

gpt-5-2-2025-12-11-thinking · CoT

Release time reported by the benchmark:

Task score
78.67%
Cost per task
0.5196 USD / Task

GPT-6 Luna (Low)

Model reported by the benchmark

As reportedOrganization: OpenAI

openai-gpt-6-luna · CoT

Release time reported by the benchmark:

Task score
37.666666666666664%
Cost per task
0.00321 USD / Task

Claude Sonnet 4.5

Model reported by the benchmark

As reportedOrganization: Anthropic

claude-sonnet-4-5-20250929 · Base LLM

Release time reported by the benchmark:

Task score
25.5%
Cost per task
0.0808 USD / Task

Claude Fable 5 (Medium)

Model reported by the benchmark

As reportedOrganization: Anthropic

anthropic-claude-fable-5 · CoT

Release time reported by the benchmark:

Task score
92.5%
Cost per task
0.6638 USD / Task

GPT-5 (Medium)

Model reported by the benchmark

As reportedOrganization: OpenAI

gpt-5-reasoning · CoT

Release time reported by the benchmark:

Task score
56.17%
Cost per task
0.3301 USD / Task

GPT-6 Luna (XHigh)

Model reported by the benchmark

As reportedOrganization: OpenAI

openai-gpt-6-luna · CoT

Release time reported by the benchmark:

Task score
73%
Cost per task
0.012172 USD / Task

Claude Opus 4.8 (High)

Model reported by the benchmark

As reportedOrganization: Anthropic

anthropic-opus-4-8 · CoT

Release time reported by the benchmark:

Task score
92%
Cost per task
1.0416 USD / Task

GPT-6.1 Sol (Max)

Model reported by the benchmark

As reportedOrganization: OpenAI

openai-gpt-6-1-sol · CoT

Release time reported by the benchmark:

Task score
96.5%
Cost per task
0.095436 USD / Task

Nemotron 3.5 Super VL (High)

Model reported by the benchmark

As reportedOrganization: NVIDIA

nvidia-nemotron-3-5-super-vl-120b-a12b · CoT

Release time reported by the benchmark:

Task score
22.666666666666668%
Cost per task
0.031481 USD / Task

Claude 3.7

Model reported by the benchmark

As reportedOrganization: Anthropic

Base LLM

Release time reported by the benchmark:

Task score
13.6%
Cost per task
0.058 USD / Task

DeepSeek V4 Pro 0813 (Low)

Model reported by the benchmark

As reportedOrganization: DeepSeek

deepseek-v4-pro-0813 · CoT

Release time reported by the benchmark:

Task score
90.5%
Cost per task
0.178231 USD / Task

Grok 4.6 (High)

Model reported by the benchmark

As reportedOrganization: xAI

xai-grok-4-6 · CoT

Release time reported by the benchmark:

Task score
87%
Cost per task
0.326866 USD / Task

Gemini 3 Deep Think (2/26)

Model reported by the benchmark

As reportedOrganization: Google

CoT

Release time reported by the benchmark:

Task score
96%
Cost per task
7.171 USD / Task

Claude Fable 5.1 (High)

Model reported by the benchmark

As reportedOrganization: Anthropic

anthropic-claude-fable-5-1 · CoT

Release time reported by the benchmark:

Task score
96%
Cost per task
0.516949 USD / Task

GPT-5.6 Terra (Low)

Model reported by the benchmark

As reportedOrganization: OpenAI

openai-gpt-5-6-terra · CoT

Release time reported by the benchmark:

Task score
60.17%
Cost per task
0.09 USD / Task

GPT-5.4 Mini (High)

Model reported by the benchmark

As reportedOrganization: OpenAI

gpt-5-4-mini-reasoning · CoT

Release time reported by the benchmark:

Task score
58%
Cost per task
0.27 USD / Task

Gemini 2.5 Flash (Preview) (Thinking 8K)

Model reported by the benchmark

As reportedOrganization: Google

gemini-2-5-flash-preview-05-20-thinking · CoT

Release time reported by the benchmark:

Task score
25.83%
Cost per task
0.1344 USD / Task

DeepSeek V4.1 Flash (Max)

Model reported by the benchmark

As reportedOrganization: DeepSeek

deepseek-v4-1-flash · CoT

Release time reported by the benchmark:

Task score
94.5%
Cost per task
0.065688 USD / Task

Claude 4.7 (High)

Model reported by the benchmark

As reportedOrganization: Anthropic

claude-4-7 · CoT

Release time reported by the benchmark:

Task score
93.5%
Cost per task
1.41 USD / Task

Claude Opus 5.5 (High)

Model reported by the benchmark

As reportedOrganization: Anthropic

anthropic-claude-opus-5-5 · CoT

Release time reported by the benchmark:

Task score
98.5%
Cost per task
0.15799 USD / Task