Average
BitextMining
Classification
Clustering
PairClassification
Reranking
Retrieval
STS
Summarization
bge-m3 0.68 0.95 0.69 0.43 0.77 0.81 0.65 0.81 0.31
distilbert-base-25lang-cased 0.43 0.65 0.46 0.37 0.69 0.34 0.10 0.53 0.31
distilbert-base-en-fr-cased 0.43 0.65 0.46 0.38 0.69 0.34 0.10 0.54 0.31
distilbert-base-fr-cased 0.41 0.45 0.46 0.38 0.69 0.34 0.10 0.54 0.31
sentence-camembert-large 0.65 0.90 0.66 0.43 0.77 0.72 0.56 0.82 0.31
sentence-flaubert-base 0.59 0.80 0.61 0.41 0.76 0.65 0.43 0.79 0.31
Solon-embeddings-base-0.1 0.64 0.95 0.67 0.43 0.76 0.78 0.41 0.78 0.31
Solon-embeddings-large-0.1 0.67 0.96 0.69 0.42 0.77 0.79 0.63 0.80 0.30
sentence-croissant-llm-base 0.62 0.91 0.65 0.43 0.77 0.68 0.52 0.76 0.29
bert-base-multilingual-cased 0.44 0.75 0.46 0.34 0.70 0.38 0.10 0.50 0.29
bert-base-multilingual-uncased 0.49 0.76 0.48 0.41 0.70 0.46 0.19 0.56 0.31
camembert-base 0.35 0.18 0.42 0.34 0.68 0.31 0.02 0.57 0.30
camembert-large 0.37 0.26 0.49 0.36 0.65 0.34 0.07 0.59 0.17
sentence-camembert-base 0.57 0.72 0.57 0.36 0.74 0.66 0.43 0.78 0.29
embed-multilingual-light-v3.0 0.63 0.89 0.61 0.39 0.74 0.76 0.55 0.78 0.31
embed-multilingual-v3.0 0.66 0.94 0.67 0.41 0.77 0.79 0.54 0.81 0.31
flaubert _base _cased 0.34 0.23 0.25 0.27 0.67 0.36 0.08 0.52 0.31
flaubert _base _uncased 0.31 0.12 0.23 0.22 0.68 0.40 0.09 0.43 0.29
flaubert _large _cased 0.27 0.11 0.25 0.25 0.65 0.30 0.01 0.33 0.29
e5-mistral-7b-instruct 0.68 0.95 0.64 0.50 0.76 0.82 0.64 0.79 0.31
multilingual-e5-base 0.65 0.95 0.65 0.43 0.75 0.75 0.56 0.78 0.31
multilingual-e5-large 0.66 0.95 0.66 0.40 0.76 0.76 0.59 0.81 0.31
multilingual-e5-small 0.63 0.94 0.60 0.39 0.75 0.73 0.52 0.78 0.32
udever-bloom-1b1 0.47 0.52 0.55 0.35 0.74 0.43 0.28 0.62 0.29
udever-bloom-560m 0.36 0.32 0.30 0.29 0.71 0.39 0.11 0.51 0.24
laser2 0.52 0.95 0.58 0.30 0.82 0.44 0.13 0.67 0.31
bge-m3-custom-fr 0.66 0.94 0.67 0.40 0.77 0.79 0.59 0.80 0.30
sentence _croissant _alpha _v0.2 0.66 0.92 0.66 0.44 0.80 0.77 0.61 0.74 0.30
sentence _croissant _alpha _v0.3 0.67 0.92 0.66 0.46 0.79 0.78 0.65 0.77 0.31
mistral-embed 0.68 0.92 0.69 0.46 0.78 0.80 0.68 0.80 0.31
LaBSE 0.59 0.96 0.65 0.39 0.74 0.61 0.33 0.74 0.30
all-MiniLM-L12-v2 0.51 0.48 0.52 0.34 0.72 0.68 0.43 0.67 0.27
all-MiniLM-L6-v2 0.50 0.40 0.52 0.35 0.71 0.65 0.38 0.68 0.28
distiluse-base-multilingual-cased-v2 0.60 0.94 0.64 0.39 0.72 0.69 0.40 0.75 0.28
multi-qa-MiniLM-L6-cos-v1 0.49 0.38 0.51 0.33 0.72 0.64 0.39 0.67 0.28
paraphrase-multilingual-MiniLM-L12-v2 0.60 0.93 0.60 0.39 0.74 0.68 0.44 0.75 0.29
paraphrase-multilingual-mpnet-base-v2 0.63 0.94 0.63 0.40 0.76 0.74 0.50 0.78 0.30
sentence-t5-base 0.59 0.83 0.58 0.41 0.72 0.70 0.45 0.75 0.30
sentence-t5-large 0.62 0.90 0.62 0.42 0.76 0.73 0.51 0.75 0.30
sentence-t5-xl 0.65 0.91 0.65 0.43 0.78 0.76 0.55 0.77 0.32
sentence-t5-xxl 0.67 0.94 0.67 0.44 0.79 0.78 0.60 0.78 0.30
text2vec-base-multilingual 0.57 0.92 0.56 0.34 0.79 0.59 0.32 0.78 0.29
text-embedding-3-large 0.71 0.96 0.74 0.48 0.80 0.86 0.73 0.81 0.30
text-embedding-3-small 0.69 0.95 0.70 0.49 0.77 0.81 0.68 0.79 0.30
text-embedding-ada-002 0.69 0.95 0.69 0.51 0.77 0.82 0.67 0.78 0.30
voyage-code-2 0.67 0.86 0.67 0.47 0.77 0.81 0.68 0.78 0.28
universal-sentence-encoder-multilingual-3 0.60 0.94 0.64 0.43 0.72 0.68 0.35 0.75 0.28
universal-sentence-encoder-multilingual-large-3 0.59 0.95 0.66 0.37 0.74 0.67 0.33 0.74 0.28
xlm-roberta-base 0.36 0.48 0.31 0.28 0.68 0.30 0.01 0.51 0.29
xlm-roberta-large 0.35 0.35 0.31 0.29 0.69 0.35 0.03 0.49 0.29
Table 9: Average performance of models per task category.
21