Project benchmark

ARC-AGI-1 · Abstract reasoning

ARC Prize · 269 configurations with published scores. Review each metric and its test conditions separately.

Report identity & conditions

Data as of2026-10-19· Report published—. See individual records for run dates and test conditions.

Leaderboard updated2026-10-15T13:43:29.213Z

Run configurations remain separate, including different reasoning settings.
Individual test dates are not given. Update dates and model release dates from the report remain separate and do not substitute for actual test dates.
Costs follow the report’s run scope and dollar amounts; they are not model API unit prices.
ARC-AGI-1 · Semi-private evaluation set. Benchmark versions are compared separately.
Task score on the semi-private evaluation set, as a percentage. It is not ranked across benchmark versions or evaluation scopes.
Cost per task as reported, in USD/task. Total costs and task counts are not invented. This is neither cost per successful task nor a current API quote.
Human reference results are shown separately and excluded from model/system rankings and cost comparisons.

Read the original report ↗

Left metricRight metric

Compare metrics within the same report. Each chart explains its direction and scope.
See the details for full values and run conditions.

Task score

Task score on the semi-private evaluation set, as a percentage. It is not ranked across benchmark versions or evaluation scopes.: showing 8 of 269 records with values. See details for configurations and dates.

Chart data · 8 entries

Name	%
Claude Fable 5 (Max)	98.5
Claude Fable 5 (XHigh)	98.5
Claude Opus 5.5 (High)	98.5
Gemini 3.8 Flash (High)	98.5
Gemini 4 Argon (High)	98.5
Gemini 4 Argon (Medium)	98.5
GPT-6 Astra (High)	98.5
GPT-6 Astra (XHigh)	98.5

Cost per task

Cost per task as reported, in USD/task. Total costs and task counts are not invented. This is neither cost per successful task nor a current API quote.: showing 8 of 269 records with values. See details for configurations and dates.

Chart data · 8 entries

Name	USD / Task
GPT-5 Nano (Minimal)	0.002
GPT-4.1-Nano	0.002
GPT-6 Luna (None)	0.002
DeepSeek V4 Flash 0731 (None)	0.002
Qwen3-235b-a22b Instruct (25/07)	0.003
GPT-6 Luna (Low)	0.003
GPT-5 Nano (Low)	0.003
Llama 4 Scout	0.004

Human reference

Human baselines are shown separately and are not ranked alongside models and systems.

Stem Grad

Task score: 98%
Cost per task: 10 USD / Task

Human Panel

Task score: 98%
Cost per task: 17 USD / Task

Full results & model versions

269 / 269configurations have published scores in this scope. Scored configurations are listed; missing values remain “Not provided.” Different run configurations of the same model are listed separately.

Grok 4.7 (Low)

Model reported by the benchmark

As reportedOrganization: xAI

xai-grok-4-7 · CoT

Release time reported by the benchmark:2026-09-21

Task score: 63.33333333333333%
Cost per task: 0.100188 USD / Task

Qwen3.8-27B (XHigh)

Model reported by the benchmark

As reportedOrganization: Alibaba

alibaba-qwen3-8-27b · CoT

Release time reported by the benchmark:2026-08-14

Task score: 87.5%
Cost per task: 0.215236 USD / Task

Icecuber

Model reported by the benchmark

As reportedOrganization: ARC Prize 2024

Kaggle · Custom

Release time reported by the benchmark:2023-11-03

Task score: 17%
Cost per task: 0.2 USD / Task

GPT-5.4 Nano (Low)

Model reported by the benchmark

As reportedOrganization: OpenAI

gpt-5-4-nano-reasoning · CoT

Release time reported by the benchmark:2026-03-17

Task score: 18.33%
Cost per task: 0.01 USD / Task

GPT-5 (Low)

Model reported by the benchmark

As reportedOrganization: OpenAI

gpt-5-reasoning · CoT

Release time reported by the benchmark:2025-08-07

Task score: 44%
Cost per task: 0.1531 USD / Task

Gemini 3.7 Flash (High)

Model reported by the benchmark

As reportedOrganization: Google

google-gemini-3-7-flash · CoT

Release time reported by the benchmark:2026-08-13

Task score: 95.5%
Cost per task: 0.120132 USD / Task

GPT-6 Luna (High)

Model reported by the benchmark

As reportedOrganization: OpenAI

openai-gpt-6-luna · CoT

Release time reported by the benchmark:2026-09-22

Task score: 70.33333333333333%
Cost per task: 0.009497 USD / Task

Inkling Small (Low)

Model reported by the benchmark

As reportedOrganization: Thinking Machines

thinky-inkling-small · CoT

Release time reported by the benchmark:2026-07-30

Task score: 52.5%
Cost per task: 0.026466 USD / Task

GPT-6 Sol (High)

Model reported by the benchmark

As reportedOrganization: OpenAI

openai-gpt-6-sol · CoT

Release time reported by the benchmark:2026-09-22

Task score: 91%
Cost per task: 0.083366 USD / Task

Gemini 3.5 Flash-Lite (High)

Model reported by the benchmark

As reportedOrganization: Google

gemini-3-5-flash-lite · CoT

Release time reported by the benchmark:2026-07-21

Task score: 53.5%
Cost per task: 0.092851 USD / Task

Inkling Small (None)

Model reported by the benchmark

As reportedOrganization: Thinking Machines

thinky-inkling-small · CoT

Release time reported by the benchmark:2026-07-30

Task score: 5.5%
Cost per task: 0.005879 USD / Task

GPT-6 Astra (Medium)

Model reported by the benchmark

As reportedOrganization: OpenAI

openai-gpt-6-astra · CoT

Release time reported by the benchmark:2026-09-02

Task score: 97.5%
Cost per task: 0.239834 USD / Task

GPT-6 Astra (High)

Model reported by the benchmark

As reportedOrganization: OpenAI

openai-gpt-6-astra · CoT

Release time reported by the benchmark:2026-09-02

Task score: 98.5%
Cost per task: 0.283703 USD / Task

Grok 4.7 (High)

Model reported by the benchmark

As reportedOrganization: xAI

xai-grok-4-7 · CoT

Release time reported by the benchmark:2026-09-21

Task score: 90.16666666666666%
Cost per task: 0.642326 USD / Task

GLM-5.3-Flash (Max)

Model reported by the benchmark

As reportedOrganization: Z.ai

zai-glm-5-3-flash · CoT

Release time reported by the benchmark:2026-08-26

Task score: 91%
Cost per task: 0.04307 USD / Task

Claude Haiku 4.5

Model reported by the benchmark

As reportedOrganization: Anthropic

claude-haiku-4-5-20251001 · Base LLM

Release time reported by the benchmark:2025-10-01

Task score: 14.33%
Cost per task: 0.0263 USD / Task

Claude Sonnet 4 (Thinking 1K)

Model reported by the benchmark

As reportedOrganization: Anthropic

claude-sonnet-4-20250514-thinking · CoT

Release time reported by the benchmark:2025-05-14

Task score: 28%
Cost per task: 0.0937 USD / Task

Grok 3

Model reported by the benchmark

As reportedOrganization: xAI

Base LLM

Release time reported by the benchmark:2025-02-19

Task score: 5.5%
Cost per task: 0.0931 USD / Task

GPT-5.4 Nano (High)

Model reported by the benchmark

As reportedOrganization: OpenAI

gpt-5-4-nano-reasoning · CoT

Release time reported by the benchmark:2026-03-17

Task score: 38.17%
Cost per task: 0.07 USD / Task

Claude 3.7 (16K)

Model reported by the benchmark

As reportedOrganization: Anthropic

Claude_3_7_thinking · CoT

Release time reported by the benchmark:2025-02-24

Task score: 28.6%
Cost per task: 0.33 USD / Task