Project benchmark

EuroEval European languages · Validation split · few-shot

EuroEval · 166 configurations with published scores. Review each metric and its test conditions separately.

Report identity & conditions

Data as of2026-10-18· Report published—. See individual records for run dates and test conditions.

Leaderboard updated2026-10-10T14:26:33+01:00

Mean rank scores cover the European generative datasets; lower is better. Overall and language-specific scores are separate. Missing scores are not zero.
Test and validation splits, and few-shot and zero-shot prompts, have separate pages. Thinking modes and model types remain part of each run’s identity; matching names do not merge configurations.
The ± value is the half-width of a 95% bootstrap confidence interval. Charts sort by point estimate, but small differences with overlapping intervals do not establish superiority.
Hugging Face models are linked only when the official URL exactly matches the catalog identifier. Short names, API prefixes and thinking modes do not establish an exact version.
Model metadata, European values and hallucination rates are outside this page’s ranking metrics. This leaderboard does not measure general ability across all tasks or languages.
The EuroEval results repository and framework use the MIT License, Copyright © Dan Saattrup Smart and Alexandra Instituttet A/S. Full terms: /licenses/euroeval.txt.
Leaderboard updated: 2026-10-10 14:26:33 CET. This is not a model release date or an individual test date.
This page covers Validation split with few-shot prompts. Model types remain attached to individual records. Per-run test dates, framework versions and a common sample count are not stated and are not inferred.

Left metricRight metric

Compare metrics within the same report. Each chart explains its direction and scope.
Error bars show 95% confidence intervals.See the details for full values and run conditions.

Mean European-language rank score

Mean rank score across the European generative tasks; lower is better. ± is the half-width of a 95% bootstrap confidence interval. Scores are not ranked across different datasets.: showing 8 of 59 records with values. See details for configurations and dates.

Chart data · 8 entries

Name	Rank score
zai-org/GLM-5.3-Flash (val)	1.32 ± 0.04 (95% CI)
deepseek/deepseek-flash#no-thinking (val)	1.45 ± 0.05 (95% CI)
Qwen/Qwen3.6-27B-FP8 (val)	1.56 ± 0.05 (95% CI)
Qwen/Qwen3.8-27B (val)	1.61 ± 0.05 (95% CI)
google/gemma-4-31B-it (val)	1.62 ± 0.05 (95% CI)
Qwen/Qwen3.5-35B-A3B-FP8 (val)	1.75 ± 0.06 (95% CI)
google/gemma-4-26B-A4B-it (val)	1.86 ± 0.06 (95% CI)
google/gemma-4-12B-it (val)	1.91 ± 0.06 (95% CI)

Albanian mean rank score

Mean rank score for Albanian; lower is better. ± is the half-width of a 95% bootstrap confidence interval. Scores are not ranked across different datasets.: showing 8 of 68 records with values. See details for configurations and dates.

Chart data · 8 entries

Name	Rank score
deepseek/deepseek-flash#no-thinking (val)	1.46 ± 0.05 (95% CI)
zai-org/GLM-5.3-Flash (val)	1.49 ± 0.06 (95% CI)
speakleash/Bielik-11B-v3.0-Instruct (val)	1.55 ± 0 (95% CI)
google/gemma-4-31B-it (val)	1.62 ± 0.06 (95% CI)
Qwen/Qwen3.5-27B-FP8 (val)	1.63 ± 0.05 (95% CI)
Qwen/Qwen3.6-27B-FP8 (val)	1.65 ± 0.07 (95% CI)
Qwen/Qwen3.8-27B (val)	1.66 ± 0.07 (95% CI)
speakleash/Bielik-Minitron-7B-v3.0-Instruct (val)	1.67 ± 0 (95% CI)

Full results & model versions

166 / 181configurations have published scores in this scope. Scored configurations are listed; missing values remain “Not provided.” Different run configurations of the same model are listed separately.

OpenLLM-France/Luciole-8B-Base (val)

Model reported by the benchmark

Validation split · few-shot · Base generative

Mean European-language rank score: 3.4 ± 0.08（95% CI）排名分
Albanian mean rank score: 3.72 ± 0.09（95% CI）排名分
Belarusian mean rank score: 3.73 ± 0.07（95% CI）排名分
Bosnian mean rank score: 3.16 ± 0.07（95% CI）排名分

More metrics

Bulgarian mean rank score: 3.25 ± 0.09（95% CI）排名分
Catalan mean rank score: 2.78 ± 0.05（95% CI）排名分
Croatian mean rank score: 3.69 ± 0.06（95% CI）排名分
Czech mean rank score: 3.33 ± 0.07（95% CI）排名分
Danish mean rank score: 3.52 ± 0.12（95% CI）排名分
Dutch mean rank score: 2.92 ± 0.19（95% CI）排名分
English mean rank score: 2.84 ± 0.05（95% CI）排名分
Estonian mean rank score: 3.55 ± 0.07（95% CI）排名分
Faroese mean rank score: 4.17 ± 0.09（95% CI）排名分
Finnish mean rank score: 3.38 ± 0.06（95% CI）排名分
French mean rank score: 2.99 ± 0.11（95% CI）排名分
German mean rank score: 3.34 ± 0.16（95% CI）排名分
Greek mean rank score: 3.68 ± 0.06（95% CI）排名分
Hungarian mean rank score: 3.68 ± 0.06（95% CI）排名分
Icelandic mean rank score: 3.67 ± 0.05（95% CI）排名分
Italian mean rank score: 3.44 ± 0.21（95% CI）排名分
Latvian mean rank score: 3.51 ± 0.07（95% CI）排名分
Lithuanian mean rank score: 3.66 ± 0.08（95% CI）排名分
Luxembourgish mean rank score: 3.47 ± 0.18（95% CI）排名分
Norwegian mean rank score: 3.43 ± 0.08（95% CI）排名分
Polish mean rank score: 3.41 ± 0.07（95% CI）排名分
Portuguese mean rank score: 3.08 ± 0.09（95% CI）排名分
Romanian mean rank score: 3.36 ± 0.07（95% CI）排名分
Serbian mean rank score: 3.58 ± 0.08（95% CI）排名分
Slovak mean rank score: 3.66 ± 0.08（95% CI）排名分
Slovene mean rank score: 3.76 ± 0.07（95% CI）排名分
Spanish mean rank score: 3.41 ± 0.13（95% CI）排名分
Swedish mean rank score: 3.35 ± 0.12（95% CI）排名分
Ukrainian mean rank score: 3.24 ± 0.1（95% CI）排名分

google/gemma-4-12B-it (val)

Model reported by the benchmark

Validation split · few-shot · Instruction-tuned

Mean European-language rank score: 1.91 ± 0.06（95% CI）排名分
Albanian mean rank score: 1.94 ± 0.08（95% CI）排名分
Belarusian mean rank score: 2.04 ± 0.05（95% CI）排名分
Bosnian mean rank score: 1.87 ± 0.03（95% CI）排名分

More metrics

Bulgarian mean rank score: 1.78 ± 0.05（95% CI）排名分
Catalan mean rank score: 2.16 ± 0.05（95% CI）排名分
Croatian mean rank score: 1.93 ± 0.06（95% CI）排名分
Czech mean rank score: 1.79 ± 0.05（95% CI）排名分
Danish mean rank score: 1.76 ± 0.04（95% CI）排名分
Dutch mean rank score: 1.85 ± 0.2（95% CI）排名分
English mean rank score: 1.46 ± 0.04（95% CI）排名分
Estonian mean rank score: 2.22 ± 0.05（95% CI）排名分
Faroese mean rank score: 2.43 ± 0.1（95% CI）排名分
Finnish mean rank score: 1.98 ± 0.05（95% CI）排名分
French mean rank score: 1.65 ± 0.15（95% CI）排名分
German mean rank score: 2.01 ± 0.17（95% CI）排名分
Greek mean rank score: 1.98 ± 0.05（95% CI）排名分
Hungarian mean rank score: 1.85 ± 0.07（95% CI）排名分
Icelandic mean rank score: 2.22 ± 0.06（95% CI）排名分
Italian mean rank score: 1.78 ± 0.17（95% CI）排名分
Latvian mean rank score: 1.71 ± 0.05（95% CI）排名分
Lithuanian mean rank score: 2.06 ± 0.08（95% CI）排名分
Luxembourgish mean rank score: 1.84 ± 0.08（95% CI）排名分
Norwegian mean rank score: 1.58 ± 0.05（95% CI）排名分
Polish mean rank score: 2.03 ± 0.05（95% CI）排名分
Portuguese mean rank score: 1.72 ± 0.07（95% CI）排名分
Romanian mean rank score: 1.82 ± 0.05（95% CI）排名分
Serbian mean rank score: 1.97 ± 0.05（95% CI）排名分
Slovak mean rank score: 2.04 ± 0.05（95% CI）排名分
Slovene mean rank score: 2.12 ± 0.06（95% CI）排名分
Spanish mean rank score: 2.36 ± 0.19（95% CI）排名分
Swedish mean rank score: 1.81 ± 0.12（95% CI）排名分
Ukrainian mean rank score: 1.8 ± 0.06（95% CI）排名分

norallm/normistral-7b-scratch (val)

Model reported by the benchmark

Validation split · few-shot · Base generative

Mean European-language rank score: Not provided
Danish mean rank score: 3.57 ± 0.08（95% CI）排名分
Faroese mean rank score: 4.51 ± 0.1（95% CI）排名分
Icelandic mean rank score: 4.1 ± 0.07（95% CI）排名分

More metrics

Albanian mean rank score: Not provided
Belarusian mean rank score: Not provided
Bosnian mean rank score: Not provided
Bulgarian mean rank score: Not provided
Catalan mean rank score: Not provided
Croatian mean rank score: Not provided
Czech mean rank score: Not provided
Dutch mean rank score: Not provided
English mean rank score: Not provided
Estonian mean rank score: Not provided
Finnish mean rank score: Not provided
French mean rank score: Not provided
German mean rank score: Not provided
Greek mean rank score: Not provided
Hungarian mean rank score: Not provided
Italian mean rank score: Not provided
Latvian mean rank score: Not provided
Lithuanian mean rank score: Not provided
Luxembourgish mean rank score: Not provided
Norwegian mean rank score: 3.52 ± 0.08（95% CI）排名分
Polish mean rank score: Not provided
Portuguese mean rank score: Not provided
Romanian mean rank score: Not provided
Serbian mean rank score: Not provided
Slovak mean rank score: Not provided
Slovene mean rank score: Not provided
Spanish mean rank score: Not provided
Swedish mean rank score: 3.47 ± 0.09（95% CI）排名分
Ukrainian mean rank score: Not provided

mii-llm/nesso-0.4B-agentic (val)

Model reported by the benchmark

Validation split · few-shot · Instruction-tuned

Mean European-language rank score: Not provided
Catalan mean rank score: 4.02 ± 0.06（95% CI）排名分
French mean rank score: 3.64 ± 0.12（95% CI）排名分
Italian mean rank score: 3.72 ± 0.16（95% CI）排名分

More metrics

Albanian mean rank score: Not provided
Belarusian mean rank score: Not provided
Bosnian mean rank score: Not provided
Bulgarian mean rank score: Not provided
Croatian mean rank score: Not provided
Czech mean rank score: Not provided
Danish mean rank score: Not provided
Dutch mean rank score: Not provided
English mean rank score: Not provided
Estonian mean rank score: Not provided
Faroese mean rank score: Not provided
Finnish mean rank score: Not provided
German mean rank score: Not provided
Greek mean rank score: Not provided
Hungarian mean rank score: Not provided
Icelandic mean rank score: Not provided
Latvian mean rank score: Not provided
Lithuanian mean rank score: Not provided
Luxembourgish mean rank score: Not provided
Norwegian mean rank score: Not provided
Polish mean rank score: Not provided
Portuguese mean rank score: 3.68 ± 0.05（95% CI）排名分
Romanian mean rank score: 3.85 ± 0.06（95% CI）排名分
Serbian mean rank score: Not provided
Slovak mean rank score: Not provided
Slovene mean rank score: Not provided
Spanish mean rank score: 3.79 ± 0.11（95% CI）排名分
Swedish mean rank score: Not provided
Ukrainian mean rank score: Not provided

OpenLLM-France/Lucie-7B-Instruct-human-data (val)

Model reported by the benchmark

Validation split · few-shot · Instruction-tuned

Mean European-language rank score: Not provided
Catalan mean rank score: 3.05 ± 0.05（95% CI）排名分
French mean rank score: 2.97 ± 0.09（95% CI）排名分
Italian mean rank score: 3.51 ± 0.15（95% CI）排名分

More metrics

Albanian mean rank score: Not provided
Belarusian mean rank score: Not provided
Bosnian mean rank score: Not provided
Bulgarian mean rank score: Not provided
Croatian mean rank score: Not provided
Czech mean rank score: Not provided
Danish mean rank score: Not provided
Dutch mean rank score: Not provided
English mean rank score: Not provided
Estonian mean rank score: Not provided
Faroese mean rank score: Not provided
Finnish mean rank score: Not provided
German mean rank score: Not provided
Greek mean rank score: Not provided
Hungarian mean rank score: Not provided
Icelandic mean rank score: Not provided
Latvian mean rank score: Not provided
Lithuanian mean rank score: Not provided
Luxembourgish mean rank score: Not provided
Norwegian mean rank score: Not provided
Polish mean rank score: Not provided
Portuguese mean rank score: 3.35 ± 0.09（95% CI）排名分
Romanian mean rank score: 3.22 ± 0.07（95% CI）排名分
Serbian mean rank score: Not provided
Slovak mean rank score: Not provided
Slovene mean rank score: Not provided
Spanish mean rank score: 3.38 ± 0.09（95% CI）排名分
Swedish mean rank score: Not provided
Ukrainian mean rank score: Not provided

nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 (val) →

Linked to an exact version in our directory

Validation split · few-shot · Reasoning model

Mean European-language rank score: 2.27 ± 0.07（95% CI）排名分
Albanian mean rank score: 2.89 ± 0.1（95% CI）排名分
Belarusian mean rank score: 2.94 ± 0.06（95% CI）排名分
Bosnian mean rank score: 2.13 ± 0.04（95% CI）排名分

More metrics

Bulgarian mean rank score: 2.15 ± 0.07（95% CI）排名分
Catalan mean rank score: 2.18 ± 0.07（95% CI）排名分
Croatian mean rank score: 2.28 ± 0.09（95% CI）排名分
Czech mean rank score: 1.92 ± 0.04（95% CI）排名分
Danish mean rank score: 1.97 ± 0.08（95% CI）排名分
Dutch mean rank score: 1.97 ± 0.21（95% CI）排名分
English mean rank score: 1.56 ± 0.03（95% CI）排名分
Estonian mean rank score: 2.85 ± 0.06（95% CI）排名分
Faroese mean rank score: 3.43 ± 0.09（95% CI）排名分
Finnish mean rank score: 2.02 ± 0.05（95% CI）排名分
French mean rank score: 1.89 ± 0.2（95% CI）排名分
German mean rank score: 2.26 ± 0.15（95% CI）排名分
Greek mean rank score: 2.33 ± 0.04（95% CI）排名分
Hungarian mean rank score: 2.38 ± 0.06（95% CI）排名分
Icelandic mean rank score: 2.91 ± 0.06（95% CI）排名分
Italian mean rank score: 1.9 ± 0.25（95% CI）排名分
Latvian mean rank score: 2.47 ± 0.06（95% CI）排名分
Lithuanian mean rank score: 2.72 ± 0.05（95% CI）排名分
Luxembourgish mean rank score: 2.8 ± 0.25（95% CI）排名分
Norwegian mean rank score: 2.06 ± 0.13（95% CI）排名分
Polish mean rank score: 1.99 ± 0.05（95% CI）排名分
Portuguese mean rank score: 1.81 ± 0.09（95% CI）排名分
Romanian mean rank score: 2.13 ± 0.07（95% CI）排名分
Serbian mean rank score: 2.37 ± 0.05（95% CI）排名分
Slovak mean rank score: 2.23 ± 0.1（95% CI）排名分
Slovene mean rank score: 2.27 ± 0.07（95% CI）排名分
Spanish mean rank score: 2.07 ± 0.14（95% CI）排名分
Swedish mean rank score: 1.91 ± 0.18（95% CI）排名分
Ukrainian mean rank score: 2.24 ± 0.07（95% CI）排名分

syvai/danskgpt-v4-31b (val)

Model reported by the benchmark

Validation split · few-shot · Instruction-tuned

Mean European-language rank score: Not provided
Danish mean rank score: 1.51 ± 0.04（95% CI）排名分

More metrics

Albanian mean rank score: Not provided
Belarusian mean rank score: Not provided
Bosnian mean rank score: Not provided
Bulgarian mean rank score: Not provided
Catalan mean rank score: Not provided
Croatian mean rank score: Not provided
Czech mean rank score: Not provided
Dutch mean rank score: Not provided
English mean rank score: Not provided
Estonian mean rank score: Not provided
Faroese mean rank score: Not provided
Finnish mean rank score: Not provided
French mean rank score: Not provided
German mean rank score: Not provided
Greek mean rank score: Not provided
Hungarian mean rank score: Not provided
Icelandic mean rank score: Not provided
Italian mean rank score: Not provided
Latvian mean rank score: Not provided
Lithuanian mean rank score: Not provided
Luxembourgish mean rank score: Not provided
Norwegian mean rank score: Not provided
Polish mean rank score: Not provided
Portuguese mean rank score: Not provided
Romanian mean rank score: Not provided
Serbian mean rank score: Not provided
Slovak mean rank score: Not provided
Slovene mean rank score: Not provided
Spanish mean rank score: Not provided
Swedish mean rank score: Not provided
Ukrainian mean rank score: Not provided

gordicaleksa/YugoGPT (val)

Model reported by the benchmark

Validation split · few-shot · Base generative

Mean European-language rank score: Not provided
Bosnian mean rank score: 2.24 ± 0.07（95% CI）排名分
Bulgarian mean rank score: 4.04 ± 0.06（95% CI）排名分
Croatian mean rank score: 2.76 ± 0.1（95% CI）排名分

More metrics

Albanian mean rank score: Not provided
Belarusian mean rank score: Not provided
Catalan mean rank score: Not provided
Czech mean rank score: 2.79 ± 0.07（95% CI）排名分
Danish mean rank score: Not provided
Dutch mean rank score: Not provided
English mean rank score: Not provided
Estonian mean rank score: Not provided
Faroese mean rank score: Not provided
Finnish mean rank score: Not provided
French mean rank score: Not provided
German mean rank score: Not provided
Greek mean rank score: Not provided
Hungarian mean rank score: Not provided
Icelandic mean rank score: Not provided
Italian mean rank score: Not provided
Latvian mean rank score: Not provided
Lithuanian mean rank score: Not provided
Luxembourgish mean rank score: Not provided
Norwegian mean rank score: Not provided
Polish mean rank score: 3.09 ± 0.08（95% CI）排名分
Portuguese mean rank score: Not provided
Romanian mean rank score: Not provided
Serbian mean rank score: 3.16 ± 0.06（95% CI）排名分
Slovak mean rank score: 3.22 ± 0.06（95% CI）排名分
Slovene mean rank score: 3.03 ± 0.09（95% CI）排名分
Spanish mean rank score: Not provided
Swedish mean rank score: Not provided
Ukrainian mean rank score: 4.24 ± 0.08（95% CI）排名分

NbAiLab/borealis-open-1b (val)

Model reported by the benchmark

Validation split · few-shot · Instruction-tuned

Mean European-language rank score: Not provided
Danish mean rank score: 3.27 ± 0.05（95% CI）排名分
Faroese mean rank score: 4.2 ± 0.09（95% CI）排名分
Icelandic mean rank score: 4.03 ± 0.07（95% CI）排名分

More metrics

Albanian mean rank score: Not provided
Belarusian mean rank score: Not provided
Bosnian mean rank score: Not provided
Bulgarian mean rank score: Not provided
Catalan mean rank score: Not provided
Croatian mean rank score: Not provided
Czech mean rank score: Not provided
Dutch mean rank score: Not provided
English mean rank score: Not provided
Estonian mean rank score: Not provided
Finnish mean rank score: Not provided
French mean rank score: Not provided
German mean rank score: Not provided
Greek mean rank score: Not provided
Hungarian mean rank score: Not provided
Italian mean rank score: Not provided
Latvian mean rank score: Not provided
Lithuanian mean rank score: Not provided
Luxembourgish mean rank score: Not provided
Norwegian mean rank score: 3.14 ± 0.06（95% CI）排名分
Polish mean rank score: Not provided
Portuguese mean rank score: Not provided
Romanian mean rank score: Not provided
Serbian mean rank score: Not provided
Slovak mean rank score: Not provided
Slovene mean rank score: Not provided
Spanish mean rank score: Not provided
Swedish mean rank score: 3.5 ± 0.1（95% CI）排名分
Ukrainian mean rank score: Not provided

INSAIT-Institute/MamayLM-Gemma-2-9B-IT-v0.1 (val)

Model reported by the benchmark

Validation split · few-shot · Instruction-tuned

Mean European-language rank score: Not provided
Bosnian mean rank score: 1.88 ± 0.05（95% CI）排名分
Bulgarian mean rank score: 1.99 ± 0.07（95% CI）排名分
Croatian mean rank score: 2.12 ± 0.08（95% CI）排名分

More metrics

Albanian mean rank score: Not provided
Belarusian mean rank score: Not provided
Catalan mean rank score: Not provided
Czech mean rank score: 1.81 ± 0.04（95% CI）排名分
Danish mean rank score: Not provided
Dutch mean rank score: Not provided
English mean rank score: Not provided
Estonian mean rank score: Not provided
Faroese mean rank score: Not provided
Finnish mean rank score: Not provided
French mean rank score: Not provided
German mean rank score: Not provided
Greek mean rank score: Not provided
Hungarian mean rank score: Not provided
Icelandic mean rank score: Not provided
Italian mean rank score: Not provided
Latvian mean rank score: Not provided
Lithuanian mean rank score: Not provided
Luxembourgish mean rank score: Not provided
Norwegian mean rank score: Not provided
Polish mean rank score: 2.36 ± 0.05（95% CI）排名分
Portuguese mean rank score: Not provided
Romanian mean rank score: Not provided
Serbian mean rank score: 2.25 ± 0.07（95% CI）排名分
Slovak mean rank score: 2.32 ± 0.05（95% CI）排名分
Slovene mean rank score: 2.31 ± 0.05（95% CI）排名分
Spanish mean rank score: Not provided
Swedish mean rank score: Not provided
Ukrainian mean rank score: 1.96 ± 0.04（95% CI）排名分

NbAiLab/borealis-270m-instruct-preview (val)

Model reported by the benchmark

Validation split · few-shot · Instruction-tuned

Mean European-language rank score: Not provided
Faroese mean rank score: 4.69 ± 0.08（95% CI）排名分

More metrics

Albanian mean rank score: Not provided
Belarusian mean rank score: Not provided
Bosnian mean rank score: Not provided
Bulgarian mean rank score: Not provided
Catalan mean rank score: Not provided
Croatian mean rank score: Not provided
Czech mean rank score: Not provided
Danish mean rank score: Not provided
Dutch mean rank score: Not provided
English mean rank score: Not provided
Estonian mean rank score: Not provided
Finnish mean rank score: Not provided
French mean rank score: Not provided
German mean rank score: Not provided
Greek mean rank score: Not provided
Hungarian mean rank score: Not provided
Icelandic mean rank score: Not provided
Italian mean rank score: Not provided
Latvian mean rank score: Not provided
Lithuanian mean rank score: Not provided
Luxembourgish mean rank score: Not provided
Norwegian mean rank score: Not provided
Polish mean rank score: Not provided
Portuguese mean rank score: Not provided
Romanian mean rank score: Not provided
Serbian mean rank score: Not provided
Slovak mean rank score: Not provided
Slovene mean rank score: Not provided
Spanish mean rank score: Not provided
Swedish mean rank score: Not provided
Ukrainian mean rank score: Not provided

kurakurai/Luth-LFM2-350M (val)

Model reported by the benchmark

Validation split · few-shot · Instruction-tuned

Mean European-language rank score: Not provided
Catalan mean rank score: 3.97 ± 0.06（95% CI）排名分

More metrics

Albanian mean rank score: Not provided
Belarusian mean rank score: Not provided
Bosnian mean rank score: Not provided
Bulgarian mean rank score: Not provided
Croatian mean rank score: Not provided
Czech mean rank score: Not provided
Danish mean rank score: Not provided
Dutch mean rank score: Not provided
English mean rank score: Not provided
Estonian mean rank score: Not provided
Faroese mean rank score: Not provided
Finnish mean rank score: Not provided
French mean rank score: Not provided
German mean rank score: Not provided
Greek mean rank score: Not provided
Hungarian mean rank score: Not provided
Icelandic mean rank score: Not provided
Italian mean rank score: Not provided
Latvian mean rank score: Not provided
Lithuanian mean rank score: Not provided
Luxembourgish mean rank score: Not provided
Norwegian mean rank score: Not provided
Polish mean rank score: Not provided
Portuguese mean rank score: Not provided
Romanian mean rank score: Not provided
Serbian mean rank score: Not provided
Slovak mean rank score: Not provided
Slovene mean rank score: Not provided
Spanish mean rank score: Not provided
Swedish mean rank score: Not provided
Ukrainian mean rank score: Not provided

utter-project/EuroLLM-1.7B (val)

Model reported by the benchmark

Validation split · few-shot · Base generative

Mean European-language rank score: Not provided
Albanian mean rank score: 3.92 ± 0.08（95% CI）排名分
Belarusian mean rank score: 3.83 ± 0.05（95% CI）排名分
Bosnian mean rank score: 3.39 ± 0.06（95% CI）排名分

More metrics

Bulgarian mean rank score: 3.3 ± 0.07（95% CI）排名分
Catalan mean rank score: 3.71 ± 0.05（95% CI）排名分
Croatian mean rank score: Not provided
Czech mean rank score: 3.58 ± 0.06（95% CI）排名分
Danish mean rank score: Not provided
Dutch mean rank score: Not provided
English mean rank score: 3.41 ± 0.07（95% CI）排名分
Estonian mean rank score: 3.59 ± 0.07（95% CI）排名分
Faroese mean rank score: Not provided
Finnish mean rank score: Not provided
French mean rank score: Not provided
German mean rank score: Not provided
Greek mean rank score: 3.39 ± 0.04（95% CI）排名分
Hungarian mean rank score: 3.8 ± 0.07（95% CI）排名分
Icelandic mean rank score: Not provided
Italian mean rank score: Not provided
Latvian mean rank score: 3.47 ± 0.07（95% CI）排名分
Lithuanian mean rank score: Not provided
Luxembourgish mean rank score: Not provided
Norwegian mean rank score: 3.57 ± 0.07（95% CI）排名分
Polish mean rank score: 3.89 ± 0.06（95% CI）排名分
Portuguese mean rank score: Not provided
Romanian mean rank score: 3.64 ± 0.07（95% CI）排名分
Serbian mean rank score: 3.9 ± 0.04（95% CI）排名分
Slovak mean rank score: 3.99 ± 0.07（95% CI）排名分
Slovene mean rank score: 3.89 ± 0.03（95% CI）排名分
Spanish mean rank score: Not provided
Swedish mean rank score: Not provided
Ukrainian mean rank score: 3.6 ± 0.07（95% CI）排名分

google/gemma-4-26B-A4B-it (val)

Model reported by the benchmark

Validation split · few-shot · Instruction-tuned

Mean European-language rank score: 1.86 ± 0.06（95% CI）排名分
Albanian mean rank score: 1.98 ± 0.08（95% CI）排名分
Belarusian mean rank score: 1.87 ± 0.06（95% CI）排名分
Bosnian mean rank score: 1.89 ± 0.03（95% CI）排名分

More metrics

Bulgarian mean rank score: 1.76 ± 0.08（95% CI）排名分
Catalan mean rank score: 2.11 ± 0.08（95% CI）排名分
Croatian mean rank score: 1.85 ± 0.06（95% CI）排名分
Czech mean rank score: 1.69 ± 0.05（95% CI）排名分
Danish mean rank score: 1.77 ± 0.05（95% CI）排名分
Dutch mean rank score: 1.86 ± 0.21（95% CI）排名分
English mean rank score: 1.42 ± 0.03（95% CI）排名分
Estonian mean rank score: 2.08 ± 0.06（95% CI）排名分
Faroese mean rank score: 2.31 ± 0.1（95% CI）排名分
Finnish mean rank score: 1.85 ± 0.05（95% CI）排名分
French mean rank score: 1.57 ± 0.17（95% CI）排名分
German mean rank score: 2.07 ± 0.17（95% CI）排名分
Greek mean rank score: 1.91 ± 0.04（95% CI）排名分
Hungarian mean rank score: 1.98 ± 0.06（95% CI）排名分
Icelandic mean rank score: 2.25 ± 0.06（95% CI）排名分
Italian mean rank score: 1.76 ± 0.23（95% CI）排名分
Latvian mean rank score: 1.69 ± 0.06（95% CI）排名分
Lithuanian mean rank score: 1.9 ± 0.06（95% CI）排名分
Luxembourgish mean rank score: 1.78 ± 0.06（95% CI）排名分
Norwegian mean rank score: 1.49 ± 0.04（95% CI）排名分
Polish mean rank score: 2 ± 0.06（95% CI）排名分
Portuguese mean rank score: 1.65 ± 0.11（95% CI）排名分
Romanian mean rank score: 1.81 ± 0.06（95% CI）排名分
Serbian mean rank score: 1.93 ± 0.05（95% CI）排名分
Slovak mean rank score: 1.98 ± 0.06（95% CI）排名分
Slovene mean rank score: 2.06 ± 0.06（95% CI）排名分
Spanish mean rank score: 2.2 ± 0.2（95% CI）排名分
Swedish mean rank score: 1.78 ± 0.09（95% CI）排名分
Ukrainian mean rank score: 1.8 ± 0.05（95% CI）排名分

CohereLabs/aya-expanse-8b (val) →

Linked to an exact version in our directory

Validation split · few-shot · Instruction-tuned

Mean European-language rank score: 2.6 ± 0.07（95% CI）排名分
Albanian mean rank score: 2.87 ± 0.07（95% CI）排名分
Belarusian mean rank score: 2.98 ± 0.06（95% CI）排名分
Bosnian mean rank score: 2.04 ± 0.04（95% CI）排名分

More metrics

Bulgarian mean rank score: 2.67 ± 0.05（95% CI）排名分
Catalan mean rank score: 2.57 ± 0.08（95% CI）排名分
Croatian mean rank score: 2.88 ± 0.08（95% CI）排名分
Czech mean rank score: 2.26 ± 0.08（95% CI）排名分
Danish mean rank score: 2.73 ± 0.09（95% CI）排名分
Dutch mean rank score: 2.31 ± 0.13（95% CI）排名分
English mean rank score: 1.86 ± 0.03（95% CI）排名分
Estonian mean rank score: 3.21 ± 0.05（95% CI）排名分
Faroese mean rank score: 3.69 ± 0.07（95% CI）排名分
Finnish mean rank score: 2.88 ± 0.06（95% CI）排名分
French mean rank score: 2.17 ± 0.14（95% CI）排名分
German mean rank score: 2.55 ± 0.16（95% CI）排名分
Greek mean rank score: 2.26 ± 0.05（95% CI）排名分
Hungarian mean rank score: 3.01 ± 0.04（95% CI）排名分
Icelandic mean rank score: 3.14 ± 0.07（95% CI）排名分
Italian mean rank score: 2.37 ± 0.17（95% CI）排名分
Latvian mean rank score: 2.85 ± 0.05（95% CI）排名分
Lithuanian mean rank score: 2.87 ± 0.07（95% CI）排名分
Luxembourgish mean rank score: 2.51 ± 0.19（95% CI）排名分
Norwegian mean rank score: 2.52 ± 0.1（95% CI）排名分
Polish mean rank score: 2.54 ± 0.05（95% CI）排名分
Portuguese mean rank score: 2.29 ± 0.07（95% CI）排名分
Romanian mean rank score: 2.08 ± 0.05（95% CI）排名分
Serbian mean rank score: 2.89 ± 0.05（95% CI）排名分
Slovak mean rank score: 2.77 ± 0.09（95% CI）排名分
Slovene mean rank score: 2.92 ± 0.06（95% CI）排名分
Spanish mean rank score: 2.43 ± 0.13（95% CI）排名分
Swedish mean rank score: 2.63 ± 0.15（95% CI）排名分
Ukrainian mean rank score: 2.34 ± 0.05（95% CI）排名分

swiss-ai/Apertus-v1.1-1.5B (val)

Model reported by the benchmark

Validation split · few-shot · Base generative

Mean European-language rank score: 3.28 ± 0.08（95% CI）排名分
Albanian mean rank score: 3.12 ± 0.09（95% CI）排名分
Belarusian mean rank score: 3.43 ± 0.08（95% CI）排名分
Bosnian mean rank score: 2.69 ± 0.06（95% CI）排名分

More metrics

Bulgarian mean rank score: 3.22 ± 0.08（95% CI）排名分
Catalan mean rank score: 3.03 ± 0.04（95% CI）排名分
Croatian mean rank score: 3.33 ± 0.07（95% CI）排名分
Czech mean rank score: 3.23 ± 0.07（95% CI）排名分
Danish mean rank score: 3.45 ± 0.05（95% CI）排名分
Dutch mean rank score: 3.22 ± 0.09（95% CI）排名分
English mean rank score: 2.95 ± 0.05（95% CI）排名分
Estonian mean rank score: 3.28 ± 0.08（95% CI）排名分
Faroese mean rank score: 3.97 ± 0.09（95% CI）排名分
Finnish mean rank score: 3.23 ± 0.06（95% CI）排名分
French mean rank score: 3.01 ± 0.12（95% CI）排名分
German mean rank score: 3.5 ± 0.1（95% CI）排名分
Greek mean rank score: 3.26 ± 0.08（95% CI）排名分
Hungarian mean rank score: 3.46 ± 0.07（95% CI）排名分
Icelandic mean rank score: 3.52 ± 0.05（95% CI）排名分
Italian mean rank score: 3.5 ± 0.15（95% CI）排名分
Latvian mean rank score: 3.09 ± 0.07（95% CI）排名分
Lithuanian mean rank score: 3.21 ± 0.06（95% CI）排名分
Luxembourgish mean rank score: 3.38 ± 0.23（95% CI）排名分
Norwegian mean rank score: 3.43 ± 0.06（95% CI）排名分
Polish mean rank score: 3.55 ± 0.08（95% CI）排名分
Portuguese mean rank score: 3.12 ± 0.12（95% CI）排名分
Romanian mean rank score: 3.18 ± 0.07（95% CI）排名分
Serbian mean rank score: 3.28 ± 0.1（95% CI）排名分
Slovak mean rank score: 3.6 ± 0.06（95% CI）排名分
Slovene mean rank score: 3.63 ± 0.06（95% CI）排名分
Spanish mean rank score: 3.34 ± 0.13（95% CI）排名分
Swedish mean rank score: 3.15 ± 0.11（95% CI）排名分
Ukrainian mean rank score: 3.4 ± 0.05（95% CI）排名分

CYFRAGOVPL/PLLuM-12B-instruct-2412 (val)

Model reported by the benchmark

Validation split · few-shot · Instruction-tuned

Mean European-language rank score: Not provided
Bosnian mean rank score: 2.21 ± 0.06（95% CI）排名分
Bulgarian mean rank score: 2.6 ± 0.07（95% CI）排名分
Croatian mean rank score: 2.8 ± 0.07（95% CI）排名分

More metrics

Albanian mean rank score: Not provided
Belarusian mean rank score: Not provided
Catalan mean rank score: Not provided
Czech mean rank score: 2.95 ± 0.07（95% CI）排名分
Danish mean rank score: Not provided
Dutch mean rank score: Not provided
English mean rank score: Not provided
Estonian mean rank score: Not provided
Faroese mean rank score: Not provided
Finnish mean rank score: Not provided
French mean rank score: Not provided
German mean rank score: Not provided
Greek mean rank score: Not provided
Hungarian mean rank score: Not provided
Icelandic mean rank score: Not provided
Italian mean rank score: Not provided
Latvian mean rank score: Not provided
Lithuanian mean rank score: Not provided
Luxembourgish mean rank score: Not provided
Norwegian mean rank score: Not provided
Polish mean rank score: 2.41 ± 0.05（95% CI）排名分
Portuguese mean rank score: Not provided
Romanian mean rank score: Not provided
Serbian mean rank score: 2.86 ± 0.09（95% CI）排名分
Slovak mean rank score: 3.01 ± 0.04（95% CI）排名分
Slovene mean rank score: 3.02 ± 0.07（95% CI）排名分
Spanish mean rank score: Not provided
Swedish mean rank score: Not provided
Ukrainian mean rank score: 2.23 ± 0.08（95% CI）排名分

INSAIT-Institute/BgGPT-Gemma-3-27B-IT (val)

Model reported by the benchmark

Validation split · few-shot · Instruction-tuned

Mean European-language rank score: Not provided
Bosnian mean rank score: 1.91 ± 0.04（95% CI）排名分
Bulgarian mean rank score: 1.7 ± 0.05（95% CI）排名分
Croatian mean rank score: 1.74 ± 0.07（95% CI）排名分

More metrics

Albanian mean rank score: Not provided
Belarusian mean rank score: Not provided
Catalan mean rank score: Not provided
Czech mean rank score: 1.68 ± 0.06（95% CI）排名分
Danish mean rank score: Not provided
Dutch mean rank score: Not provided
English mean rank score: Not provided
Estonian mean rank score: Not provided
Faroese mean rank score: Not provided
Finnish mean rank score: Not provided
French mean rank score: Not provided
German mean rank score: Not provided
Greek mean rank score: Not provided
Hungarian mean rank score: Not provided
Icelandic mean rank score: Not provided
Italian mean rank score: Not provided
Latvian mean rank score: Not provided
Lithuanian mean rank score: Not provided
Luxembourgish mean rank score: Not provided
Norwegian mean rank score: Not provided
Polish mean rank score: 2 ± 0.05（95% CI）排名分
Portuguese mean rank score: Not provided
Romanian mean rank score: Not provided
Serbian mean rank score: 2.01 ± 0.06（95% CI）排名分
Slovak mean rank score: 2.05 ± 0.06（95% CI）排名分
Slovene mean rank score: 1.94 ± 0.08（95% CI）排名分
Spanish mean rank score: Not provided
Swedish mean rank score: Not provided
Ukrainian mean rank score: 1.77 ± 0.04（95% CI）排名分

CYFRAGOVPL/PLLuM-12B-nc-chat-2412 (val)

Model reported by the benchmark

Validation split · few-shot · Instruction-tuned

Mean European-language rank score: Not provided
Belarusian mean rank score: 3.3 ± 0.06（95% CI）排名分
Bosnian mean rank score: 2.72 ± 0.07（95% CI）排名分
Bulgarian mean rank score: 3.25 ± 0.05（95% CI）排名分

More metrics

Albanian mean rank score: Not provided
Catalan mean rank score: Not provided
Croatian mean rank score: 3.33 ± 0.06（95% CI）排名分
Czech mean rank score: 3.12 ± 0.08（95% CI）排名分
Danish mean rank score: Not provided
Dutch mean rank score: Not provided
English mean rank score: Not provided
Estonian mean rank score: Not provided
Faroese mean rank score: Not provided
Finnish mean rank score: Not provided
French mean rank score: Not provided
German mean rank score: Not provided
Greek mean rank score: Not provided
Hungarian mean rank score: Not provided
Icelandic mean rank score: Not provided
Italian mean rank score: Not provided
Latvian mean rank score: Not provided
Lithuanian mean rank score: Not provided
Luxembourgish mean rank score: Not provided
Norwegian mean rank score: Not provided
Polish mean rank score: 2.56 ± 0.06（95% CI）排名分
Portuguese mean rank score: Not provided
Romanian mean rank score: Not provided
Serbian mean rank score: 3.03 ± 0.08（95% CI）排名分
Slovak mean rank score: 2.98 ± 0.06（95% CI）排名分
Slovene mean rank score: 3.37 ± 0.07（95% CI）排名分
Spanish mean rank score: Not provided
Swedish mean rank score: Not provided
Ukrainian mean rank score: 3.48 ± 0.08（95% CI）排名分

mii-llm/zagreus-0.4B-ita (val)

Model reported by the benchmark

Validation split · few-shot · Base generative

Mean European-language rank score: Not provided
Catalan mean rank score: 4.22 ± 0.07（95% CI）排名分
French mean rank score: 4.12 ± 0.11（95% CI）排名分
Italian mean rank score: 4.08 ± 0.11（95% CI）排名分

More metrics

Albanian mean rank score: Not provided
Belarusian mean rank score: Not provided
Bosnian mean rank score: Not provided
Bulgarian mean rank score: Not provided
Croatian mean rank score: Not provided
Czech mean rank score: Not provided
Danish mean rank score: Not provided
Dutch mean rank score: Not provided
English mean rank score: Not provided
Estonian mean rank score: Not provided
Faroese mean rank score: Not provided
Finnish mean rank score: Not provided
German mean rank score: Not provided
Greek mean rank score: Not provided
Hungarian mean rank score: Not provided
Icelandic mean rank score: Not provided
Latvian mean rank score: Not provided
Lithuanian mean rank score: Not provided
Luxembourgish mean rank score: Not provided
Norwegian mean rank score: Not provided
Polish mean rank score: Not provided
Portuguese mean rank score: 4.17 ± 0.05（95% CI）排名分
Romanian mean rank score: 4.3 ± 0.06（95% CI）排名分
Serbian mean rank score: Not provided
Slovak mean rank score: Not provided
Slovene mean rank score: Not provided
Spanish mean rank score: 4.24 ± 0.07（95% CI）排名分
Swedish mean rank score: Not provided
Ukrainian mean rank score: Not provided