Flores_fr-en
Flores_en-fr
DiaBla_fr-en
STSBenchmarkMultilingual
STS22
SICKFr
SummEvalFr
BitextMining STS Summarization
bge-m3 1.00 1.00 0.85 0.82 0.82 0.78 0.31
distilbert-base-25lang-cased 0.92 0.91 0.11 0.57 0.41 0.62 0.31
distilbert-base-en-fr-cased 0.92 0.91 0.11 0.57 0.42 0.62 0.31
distilbert-base-fr-cased 0.63 0.65 0.06 0.57 0.43 0.62 0.31
sentence-camembert-large 0.99 1.00 0.70 0.86 0.82 0.78 0.31
sentence-flaubert-base 0.96 0.97 0.47 0.86 0.74 0.78 0.31
Solon-embeddings-base-0.1 1.00 1.00 0.85 0.79 0.81 0.75 0.31
Solon-embeddings-large-0.1 1.00 1.00 0.87 0.80 0.83 0.77 0.30
sentence-croissant-llm-base 1.00 1.00 0.74 0.79 0.79 0.70 0.29
bert-base-multilingual-cased 0.97 0.98 0.30 0.52 0.39 0.59 0.29
bert-base-multilingual-uncased 0.95 0.98 0.36 0.55 0.56 0.58 0.31
camembert-base 0.26 0.25 0.04 0.55 0.61 0.54 0.30
sentence-camembert-base 0.90 0.90 0.36 0.82 0.78 0.74 0.29
sentence-camembert-large 0.99 1.00 0.68 0.86 0.82 0.78 0.31
embed-multilingual-light-v3.0 1.00 1.00 0.66 0.76 0.83 0.76 0.31
embed-multilingual-v3.0 1.00 1.00 0.83 0.82 0.83 0.79 0.31
flaubert_base_cased 0.31 0.36 0.02 0.37 0.65 0.54 0.31
flaubert_base_uncased 0.25 0.08 0.03 0.33 0.55 0.42 0.29
flaubert_large_cased 0.15 0.17 0.01 0.16 0.49 0.35 0.29
e5-mistral-7b-instruct 1.00 1.00 0.85 0.83 0.76 0.79 0.31
multilingual-e5-base 1.00 1.00 0.85 0.81 0.78 0.76 0.31
multilingual-e5-large 1.00 1.00 0.85 0.83 0.80 0.79 0.31
multilingual-e5-small 1.00 1.00 0.82 0.79 0.80 0.76 0.32
udever-bloom-1b1 0.75 0.78 0.03 0.50 0.77 0.60 0.29
udever-bloom-560m 0.50 0.37 0.08 0.37 0.61 0.55 0.24
laser2 1.00 1.00 0.86 0.70 0.65 0.65 0.31
bge-m3-custom-fr 1.00 1.00 0.83 0.81 0.82 0.76 0.30
sentence_croissant_alpha_v0.2 1.00 1.00 0.75 0.73 0.79 0.69 0.30
sentence_croissant_alpha_v0.3 1.00 1.00 0.77 0.78 0.81 0.72 0.31
mistral-embed 1.00 1.00 0.75 0.80 0.83 0.76 0.31
LaBSE 1.00 1.00 0.88 0.75 0.78 0.70 0.30
all-MiniLM-L12-v2 0.71 0.62 0.10 0.67 0.70 0.63 0.27
all-MiniLM-L6-v2 0.62 0.56 0.03 0.65 0.77 0.62 0.28
distiluse-base-multilingual-cased-v2 1.00 1.00 0.83 0.77 0.76 0.72 0.28
multi-qa-MiniLM-L6-cos-v1 0.55 0.50 0.09 0.64 0.75 0.62 0.28
paraphrase-multilingual-MiniLM-L12-v2 1.00 1.00 0.78 0.80 0.71 0.75 0.29
paraphrase-multilingual-mpnet-base-v2 1.00 1.00 0.81 0.85 0.74 0.76 0.30
sentence-t5-base 0.97 0.96 0.55 0.74 0.78 0.72 0.30
sentence-t5-large 0.99 0.99 0.71 0.78 0.75 0.73 0.30
sentence-t5-xl 0.99 0.99 0.76 0.79 0.77 0.75 0.32
sentence-t5-xxl 1.00 1.00 0.83 0.81 0.77 0.77 0.30
text2vec-base-multilingual 0.99 0.99 0.78 0.83 0.74 0.77 0.29
text-embedding-3-large 1.00 1.00 0.88 0.83 0.82 0.79 0.30
text-embedding-3-small 1.00 1.00 0.86 0.81 0.81 0.76 0.30
text-embedding-ada-002 0.99 0.99 0.86 0.78 0.81 0.76 0.30
voyage-code-2 1.00 0.99 0.60 0.79 0.80 0.74 0.28
universal-sentence-encoder-multilingual-3 1.00 1.00 0.82 0.75 0.78 0.71 0.28
universal-sentence-encoder-multilingual-large-3 1.00 1.00 0.84 0.78 0.71 0.74 0.28
xlm-roberta-base 0.70 0.53 0.21 0.46 0.57 0.49 0.29
xlm-roberta-large 0.65 0.26 0.13 0.42 0.55 0.50 0.29
Table 12: Performance of each model for Bitext Mining, Semantic Textual Similarity (STS) and Summarization.
24