Project benchmark

ARC-AGI-3 · standard framework

ARC Prize · 68 configurations with published scores. Review each metric and its test conditions separately.

Report identity & conditions

Data as of2026-10-28· Report published—. See individual records for run dates and test conditions.

Leaderboard updated2026-10-21T15:42:36.395Z

Run configurations remain separate, including different reasoning settings.
Individual test dates are not given. Update dates and model release dates from the report remain separate and do not substitute for actual test dates.
Costs follow the report’s run scope and dollar amounts; they are not model API unit prices.
ARC-AGI-3 · Semi-private evaluation set. Benchmark versions are compared separately.
Learning efficiency for interactive tasks. A score of 100% matches human efficiency at completing the games; this is not static-question accuracy.
Total evaluation cost in USD. It is not converted into cost per task and is not a current API quote.
The standard harness uses a minimal, provider-neutral interface and allows the model to keep notes.

Left metricRight metric

Compare metrics within the same report. Each chart explains its direction and scope.
See the details for full values and run conditions.

Skill acquisition efficiency

Learning efficiency for interactive tasks. A score of 100% matches human efficiency at completing the games; this is not static-question accuracy.: showing 8 of 68 records with values. See details for configurations and dates.

Chart data · 8 entries

Name	%
GPT-6 Astra (Max)	62.713
GPT-6 Astra (XHigh)	59.343
GPT-6 Astra (High)	54.819
GPT-6.1 Sol (Max)	52.729
Gemini 4 Argon (Low)	49.794
Gemini 4 Argon (Medium)	46.266
GPT-6.1 Sol (XHigh)	39.929
GPT-6 Astra (Medium)	38.588

Total run cost

Total evaluation cost in USD. It is not converted into cost per task and is not a current API quote.: showing 8 of 68 records with values. See details for configurations and dates.

Chart data · 8 entries

Name	USD
GPT-6 Luna (Low)	228.833
GPT-6 Luna (None)	229.771
GPT-6 Luna (Medium)	241.28
GPT-6 Luna (High)	255.979
GPT-6 Luna (Max)	262.238
GPT-6 Luna (XHigh)	262.684
Claude Haiku 5.5 (Max)	388.106
DeepSeek V4.1 Flash (None)	734.725

Full results & model versions

68 / 68configurations have published scores in this scope. Scored configurations are listed; missing values remain “Not provided.” Different run configurations of the same model are listed separately.

Grok 4.5 (High)

Model reported by the benchmark

Standard framework

As reportedOrganization: xAI

xai-grok-4-5 · CoT

Release time reported by the benchmark:2026-07-16

Skill acquisition efficiency: 0.3%
Total run cost: 6892.87 USD

Grok 4.6 (XHigh)

Model reported by the benchmark

Standard framework

As reportedOrganization: xAI

xai-grok-4-6 · CoT

Release time reported by the benchmark:2026-08-11

Skill acquisition efficiency: 2.11%
Total run cost: 5612 USD

GPT-6 Astra (Medium)

Model reported by the benchmark

Standard framework

As reportedOrganization: OpenAI

openai-gpt-6-astra · CoT

Release time reported by the benchmark:2026-09-02

Skill acquisition efficiency: 38.58757449733784%
Total run cost: 48090.34598999999 USD

Gemini 4 Argon (Medium)

Model reported by the benchmark

Standard framework

As reportedOrganization: Google

google-gemini-4-argon · CoT

Release time reported by the benchmark:2026-10-13

Skill acquisition efficiency: 46.265668402958575%
Total run cost: 6179.253994 USD

Gemini 3.8 Flash (Low)

Model reported by the benchmark

Standard framework

As reportedOrganization: Google

google-gemini-3-8-flash · CoT

Release time reported by the benchmark:2026-09-02

Skill acquisition efficiency: 5.989560493697216%
Total run cost: 2669.05677675 USD

Opus 4.7 (High)

Model reported by the benchmark

Standard framework

As reportedOrganization: Anthropic

anthropic-opus-4-7 · CoT

Release time reported by the benchmark:2026-04-16

Skill acquisition efficiency: 0.18%
Total run cost: 10000 USD

Grok 4.5 (Low)

Model reported by the benchmark

Standard framework

As reportedOrganization: xAI

xai-grok-4-5 · CoT

Release time reported by the benchmark:2026-07-16

Skill acquisition efficiency: 0.26%
Total run cost: 8718.48 USD

GPT-6 Astra (Low)

Model reported by the benchmark

Standard framework

As reportedOrganization: OpenAI

openai-gpt-6-astra · CoT

Release time reported by the benchmark:2026-09-02

Skill acquisition efficiency: 17.452190401133805%
Total run cost: 38166.47030000001 USD

DeepSeek V4.1 Flash (High)

Model reported by the benchmark

Standard framework

As reportedOrganization: DeepSeek

deepseek-v4-1-flash · CoT

Release time reported by the benchmark:2026-09-10

Skill acquisition efficiency: 0.4846664966204641%
Total run cost: 1005.7964823 USD

GPT-5.6 Terra (Low)

Model reported by the benchmark

Standard framework

As reportedOrganization: OpenAI

openai-gpt-5-6-terra · CoT

Release time reported by the benchmark:2026-07-09

Skill acquisition efficiency: 0.01%
Total run cost: 5068.48 USD

Gemini 3.8 Flash (High)

Model reported by the benchmark

Standard framework

As reportedOrganization: Google

google-gemini-3-8-flash · CoT

Release time reported by the benchmark:2026-09-02

Skill acquisition efficiency: 10.36627922392312%
Total run cost: 4398.2756895 USD

GPT-6 Luna (Low)

Model reported by the benchmark

Standard framework

As reportedOrganization: OpenAI

openai-gpt-6-luna · CoT

Release time reported by the benchmark:2026-09-22

Skill acquisition efficiency: 0.03291205821227175%
Total run cost: 228.8330075 USD

Grok 4.7 (High)

Model reported by the benchmark

Standard framework

As reportedOrganization: xAI

xai-grok-4-7 · CoT

Release time reported by the benchmark:2026-09-21

Skill acquisition efficiency: 1.7332202934667684%
Total run cost: 3741.649478 USD

GPT-6.1 Sol (High)

Model reported by the benchmark

Standard framework

As reportedOrganization: OpenAI

openai-gpt-6-1-sol · CoT

Release time reported by the benchmark:2026-09-29

Skill acquisition efficiency: 26.71728484425706%
Total run cost: 9757.61782 USD

GPT-6 Luna (High)

Model reported by the benchmark

Standard framework

As reportedOrganization: OpenAI

openai-gpt-6-luna · CoT

Release time reported by the benchmark:2026-09-22

Skill acquisition efficiency: 0.18024216987988248%
Total run cost: 255.97928910000002 USD

Gemini 3.8 Flash (Medium)

Model reported by the benchmark

Standard framework

As reportedOrganization: Google

google-gemini-3-8-flash · CoT

Release time reported by the benchmark:2026-09-02

Skill acquisition efficiency: 3.9878093830617267%
Total run cost: 2727.66378825 USD

GPT-6 Luna (XHigh)

Model reported by the benchmark

Standard framework

As reportedOrganization: OpenAI

openai-gpt-6-luna · CoT

Release time reported by the benchmark:2026-09-22

Skill acquisition efficiency: 0.16413333960356602%
Total run cost: 262.6844075 USD

Claude Opus 4.8 (High)

Model reported by the benchmark

Standard framework

As reportedOrganization: Anthropic

anthropic-opus-4-8 · CoT

Release time reported by the benchmark:2026-06-01

Skill acquisition efficiency: 1.52%
Total run cost: 10000 USD

Grok 4.7 (Medium)

Model reported by the benchmark

Standard framework

As reportedOrganization: xAI

xai-grok-4-7 · CoT

Release time reported by the benchmark:2026-09-21

Skill acquisition efficiency: 1.7056643767835524%
Total run cost: 3795.4947700000002 USD

Claude Sonnet 5.5 (Medium)

Model reported by the benchmark

Standard framework

As reportedOrganization: Anthropic

anthropic-claude-sonnet-5-5 · CoT

Release time reported by the benchmark:2026-09-28

Skill acquisition efficiency: 0.3961083341784243%
Total run cost: 4839.68249 USD