Project benchmark

EuroEval European languages · Validation split · zero-shot

EuroEval · 62 configurations with published scores. Review each metric and its test conditions separately.

Report identity & conditions

Data as of2026-10-18· Report published—. See individual records for run dates and test conditions.

Leaderboard updated2026-10-10T14:26:33+01:00

Mean rank scores cover the European generative datasets; lower is better. Overall and language-specific scores are separate. Missing scores are not zero.
Test and validation splits, and few-shot and zero-shot prompts, have separate pages. Thinking modes and model types remain part of each run’s identity; matching names do not merge configurations.
The ± value is the half-width of a 95% bootstrap confidence interval. Charts sort by point estimate, but small differences with overlapping intervals do not establish superiority.
Hugging Face models are linked only when the official URL exactly matches the catalog identifier. Short names, API prefixes and thinking modes do not establish an exact version.
Model metadata, European values and hallucination rates are outside this page’s ranking metrics. This leaderboard does not measure general ability across all tasks or languages.
The EuroEval results repository and framework use the MIT License, Copyright © Dan Saattrup Smart and Alexandra Instituttet A/S. Full terms: /licenses/euroeval.txt.
Leaderboard updated: 2026-10-10 14:26:33 CET. This is not a model release date or an individual test date.
This page covers Validation split with zero-shot prompts. Model types remain attached to individual records. Per-run test dates, framework versions and a common sample count are not stated and are not inferred.

Left metricRight metric

Compare metrics within the same report. Each chart explains its direction and scope.
Error bars show 95% confidence intervals.See the details for full values and run conditions.

Mean European-language rank score

Mean rank score across the European generative tasks; lower is better. ± is the half-width of a 95% bootstrap confidence interval. Scores are not ranked across different datasets.: showing 8 of 36 records with values. See details for configurations and dates.

Chart data · 8 entries

Name	Rank score
gpt-6.1-sol (zero-shot, val)	1.29 ± 0.04 (95% CI)
gpt-6-astra (zero-shot, val)	1.3 ± 0.04 (95% CI)
gemini/gemini-3.7-flash (zero-shot, val)	1.34 ± 0.04 (95% CI)
gemini/gemini-3.6-flash (zero-shot, val)	1.42 ± 0.04 (95% CI)
gpt-6-sol (zero-shot, val)	1.44 ± 0.03 (95% CI)
gpt-5.6-sol (zero-shot, val)	1.46 ± 0.04 (95% CI)
gemini/gemini-3-flash-preview#thinking (zero-shot, val)	1.48 ± 0.03 (95% CI)
deepseek/deepseek-flash#low (zero-shot, val)	1.51 ± 0.03 (95% CI)

Albanian mean rank score

Mean rank score for Albanian; lower is better. ± is the half-width of a 95% bootstrap confidence interval. Scores are not ranked across different datasets.: showing 8 of 39 records with values. See details for configurations and dates.

Chart data · 8 entries

Name	Rank score
gemini/gemini-3.1-pro-preview (zero-shot, val)	1.32 ± 0.05 (95% CI)
gpt-6-astra (zero-shot, val)	1.36 ± 0.06 (95% CI)
gemini/gemini-3.7-flash (zero-shot, val)	1.4 ± 0.06 (95% CI)
gpt-6.1-sol (zero-shot, val)	1.42 ± 0.07 (95% CI)
gemini/gemini-3.6-flash (zero-shot, val)	1.48 ± 0.05 (95% CI)
gpt-5.6-sol (zero-shot, val)	1.49 ± 0.07 (95% CI)
gpt-6-sol (zero-shot, val)	1.55 ± 0.08 (95% CI)
gpt-5.6-terra (zero-shot, val)	1.6 ± 0.06 (95% CI)

Full results & model versions

62 / 90configurations have published scores in this scope. Scored configurations are listed; missing values remain “Not provided.” Different run configurations of the same model are listed separately.