Project benchmark

ARC-AGI-3 · Provider Adapter

ARC Prize · 44 configurations with published scores. Review each metric and its test conditions separately.

Report identity & conditions

Data as of2026-11-03· Report published—. See individual records for run dates and test conditions.

Leaderboard updated2026-11-02T14:21:29.499Z

Run configurations remain separate, including different reasoning settings.
Individual test dates are not given. Update dates and model release dates from the report remain separate and do not substitute for actual test dates.
Costs follow the report’s run scope and dollar amounts; they are not model API unit prices.
ARC-AGI-3 · Semi-private evaluation set. Benchmark versions are compared separately.
Learning efficiency for interactive tasks. A score of 100% matches human efficiency at completing the games; this is not static-question accuracy.
Total evaluation cost in USD. It is not converted into cost per task and is not a current API quote.
Provider Adapter uses the provider’s context-management design and is compared separately from the standard harness.

Left metricRight metric

Compare metrics within the same report. Each chart explains its direction and scope.
See the details for full values and run conditions.

Skill acquisition efficiency

Learning efficiency for interactive tasks. A score of 100% matches human efficiency at completing the games; this is not static-question accuracy.: showing 8 of 44 records with values. See details for configurations and dates.

Chart data · 8 entries

Name	%
GPT-6 Astra - Provider Adapter (High)	99.946
GPT-6 Astra - Provider Adapter (Max)	98.551
GPT-6 Astra - Provider Adapter (XHigh)	98.442
GPT-6 Astra - Provider Adapter (Medium)	98.439
GPT-6 Astra - Provider Adapter (Low)	98.028
Gemini 4 Argon - Provider Adapter (Low)	97.459
Gemini 4 Argon - Provider Adapter (Medium)	97.345
Gemini 4 Argon - Provider Adapter (High)	97.304

Total run cost

Total evaluation cost in USD. It is not converted into cost per task and is not a current API quote.: showing 8 of 44 records with values. See details for configurations and dates.

Chart data · 8 entries

Name	USD
GPT-6 Luna - Provider Adapter (None)	205.032
GPT-6 Luna - Provider Adapter (Medium)	213.72
GPT-6 Luna - Provider Adapter (Low)	214.291
GPT-6 Luna - Provider Adapter (XHigh)	223.172
GPT-6 Luna - Provider Adapter (High)	223.895
GPT-6 Luna - Provider Adapter (Max)	237.143
Claude Haiku 5.5 - Provider Adapter (Max)	437.141
DeepSeek V4.1 Flash - Provider Adapter (None)	648.412

Full results & model versions

44 / 44configurations have published scores in this scope. Scored configurations are listed; missing values remain “Not provided.” Different run configurations of the same model are listed separately.

GPT-6 Luna - Provider Adapter (Max)

Model reported by the benchmark

Provider Adapter framework

As reportedOrganization: OpenAI

openai-gpt-6-luna-provider-adapter · CoT

Release time reported by the benchmark:2026-09-22

Skill acquisition efficiency: 0.5868386891554969%
Total run cost: 237.1428722 USD

GPT-6 Sol - Provider Adapter (None)

Model reported by the benchmark

Provider Adapter framework

As reportedOrganization: OpenAI

openai-gpt-6-sol-provider-adapter · CoT

Release time reported by the benchmark:2026-09-22

Skill acquisition efficiency: 1.2059164341155284%
Total run cost: 4233.770312 USD

DeepSeek V4.1 Flash - Provider Adapter (High)

Model reported by the benchmark

Provider Adapter framework

As reportedOrganization: DeepSeek

deepseek-v4-1-flash-provider-adapter · CoT

Release time reported by the benchmark:2026-09-10

Skill acquisition efficiency: 3.1518486978853844%
Total run cost: 1021.5134985 USD

DeepSeek V4.1 Flash 1017 - Provider Adapter (Max)

Model reported by the benchmark

Provider Adapter framework

As reportedOrganization: DeepSeek

deepseek-v4-1-flash-1017-provider-adapter · CoT

Release time reported by the benchmark:2026-10-17

Skill acquisition efficiency: 5.80827100843473%
Total run cost: 1179.1379751 USD

GPT-6.1 Sol - Provider Adapter (XHigh)

Model reported by the benchmark

Provider Adapter framework

As reportedOrganization: OpenAI

openai-gpt-6-1-sol-provider-adapter · CoT

Release time reported by the benchmark:2026-09-29

Skill acquisition efficiency: 96.37493342875182%
Total run cost: 4360.206418 USD

Grok 4.7 - Provider Adapter (Low)

Model reported by the benchmark

Provider Adapter framework

As reportedOrganization: xAI

xai-grok-4-7-provider-adapter · CoT

Release time reported by the benchmark:2026-09-21

Skill acquisition efficiency: 0.7739582026314942%
Total run cost: 5753.573154 USD

Claude Sonnet 5.5 - Provider Adapter (Medium)

Model reported by the benchmark

Provider Adapter framework

As reportedOrganization: Anthropic

anthropic-claude-sonnet-5-5-provider-adapter · CoT

Release time reported by the benchmark:2026-09-28

Skill acquisition efficiency: 4.887554909249332%
Total run cost: 5250.163516 USD

GPT-6.1 Sol - Provider Adapter (Low)

Model reported by the benchmark

Provider Adapter framework

As reportedOrganization: OpenAI

openai-gpt-6-1-sol-provider-adapter · CoT

Release time reported by the benchmark:2026-09-29

Skill acquisition efficiency: 82.78706730491692%
Total run cost: 6622.199092 USD

GPT-6 Sol - Provider Adapter (Max)

Model reported by the benchmark

Provider Adapter framework

As reportedOrganization: OpenAI

openai-gpt-6-sol-provider-adapter · CoT

Release time reported by the benchmark:2026-09-22

Skill acquisition efficiency: 23.028056734101497%
Total run cost: 8721.949674 USD

Claude Sonnet 5.5 - Provider Adapter (High)

Model reported by the benchmark

Provider Adapter framework

As reportedOrganization: Anthropic

anthropic-claude-sonnet-5-5-provider-adapter · CoT

Release time reported by the benchmark:2026-09-28

Skill acquisition efficiency: 35.47664601535195%
Total run cost: 10244.13243 USD

Gemini 4 Argon - Provider Adapter (Low)

Model reported by the benchmark

Provider Adapter framework

As reportedOrganization: Google

google-gemini-4-argon-provider-adapter · CoT

Release time reported by the benchmark:2026-10-13

Skill acquisition efficiency: 97.4591718377776%
Total run cost: 4770.148614 USD

GPT-6 Sol - Provider Adapter (Low)

Model reported by the benchmark

Provider Adapter framework

As reportedOrganization: OpenAI

openai-gpt-6-sol-provider-adapter · CoT

Release time reported by the benchmark:2026-09-22

Skill acquisition efficiency: 1.7619154311646184%
Total run cost: 4745.286212 USD

GPT-6 Astra - Provider Adapter (Max)

Model reported by the benchmark

Provider Adapter framework

As reportedOrganization: OpenAI

openai-gpt-6-astra-provider-adapter · CoT

Release time reported by the benchmark:2026-09-02

Skill acquisition efficiency: 98.55074288592294%
Total run cost: 17331.95327 USD

GPT-6 Astra - Provider Adapter (None)

Model reported by the benchmark

Provider Adapter framework

As reportedOrganization: OpenAI

openai-gpt-6-astra-provider-adapter · CoT

Release time reported by the benchmark:2026-09-02

Skill acquisition efficiency: 96.72021831785531%
Total run cost: 23456.663940000002 USD

GPT-6 Luna - Provider Adapter (Medium)

Model reported by the benchmark

Provider Adapter framework

As reportedOrganization: OpenAI

openai-gpt-6-luna-provider-adapter · CoT

Release time reported by the benchmark:2026-09-22

Skill acquisition efficiency: 0.3161200923817726%
Total run cost: 213.71995940000002 USD

Gemini 4 Argon - Provider Adapter (High)

Model reported by the benchmark

Provider Adapter framework

As reportedOrganization: Google

google-gemini-4-argon-provider-adapter · CoT

Release time reported by the benchmark:2026-10-13

Skill acquisition efficiency: 97.30432688837197%
Total run cost: 4625.84128 USD

Claude Sonnet 5.5 - Provider Adapter (Low)

Model reported by the benchmark

Provider Adapter framework

As reportedOrganization: Anthropic

anthropic-claude-sonnet-5-5-provider-adapter · CoT

Release time reported by the benchmark:2026-09-28

Skill acquisition efficiency: 0.6244567651378714%
Total run cost: 4796.883118 USD

GPT-6 Astra - Provider Adapter (XHigh)

Model reported by the benchmark

Provider Adapter framework

As reportedOrganization: OpenAI

openai-gpt-6-astra-provider-adapter · CoT

Release time reported by the benchmark:2026-09-02

Skill acquisition efficiency: 98.44155844155843%
Total run cost: 18147.191119999996 USD

Grok 4.7 - Provider Adapter (XHigh)

Model reported by the benchmark

Provider Adapter framework

As reportedOrganization: xAI

xai-grok-4-7-provider-adapter · CoT

Release time reported by the benchmark:2026-09-21

Skill acquisition efficiency: 7.01648689583893%
Total run cost: 4575.921274 USD

GPT-6 Astra - Provider Adapter (High)

Model reported by the benchmark

Provider Adapter framework

As reportedOrganization: OpenAI

openai-gpt-6-astra-provider-adapter · CoT

Release time reported by the benchmark:2026-09-02

Skill acquisition efficiency: 99.945712321257%
Total run cost: 18816.63138 USD