MasakhaNEWSS2S
MasakhaNEWSP2P
MLSUMS2S
MLSUMP2P
HALS2S
AlloProfS2S
AlloProfP2P
Clustering
bge-m3 0.42 0.45 0.44 0.43 0.31 0.37 0.59
distilbert-base-25lang-cased 0.33 0.32 0.31 0.41 0.24 0.43 0.57
distilbert-base-en-fr-cased 0.34 0.34 0.31 0.41 0.25 0.42 0.57
distilbert-base-fr-cased 0.35 0.34 0.31 0.41 0.24 0.43 0.57
sentence-camembert-large 0.37 0.44 0.43 0.43 0.32 0.40 0.62
sentence-flaubert-base 0.30 0.49 0.41 0.41 0.32 0.40 0.57
Solon-embeddings-base-0.1 0.36 0.50 0.42 0.43 0.30 0.37 0.61
Solon-embeddings-large-0.1 0.31 0.46 0.43 0.43 0.32 0.37 0.63
sentence-croissant-llm-base 0.41 0.54 0.34 0.43 0.29 0.33 0.64
bert-base-multilingual-cased 0.24 0.24 0.32 0.41 0.25 0.43 0.51
bert-base-multilingual-uncased 0.42 0.50 0.31 0.43 0.26 0.35 0.61
camembert-base 0.27 0.44 0.27 0.41 0.16 0.29 0.54
camembert-large 0.33 0.42 0.35 0.44 0.03 0.34 0.59
sentence-camembert-base 0.31 0.36 0.27 0.36 0.25 0.39 0.59
embed-multilingual-light-v3.0 0.29 0.57 0.33 0.43 0.20 0.31 0.62
embed-multilingual-v3.0 0.32 0.53 0.35 0.45 0.24 0.36 0.64
flaubert_base_cased 0.21 0.42 0.17 0.39 0.04 0.14 0.53
flaubert_base_uncased 0.23 0.28 0.15 0.33 0.02 0.13 0.43
flaubert_large_cased 0.25 0.26 0.19 0.38 0.07 0.22 0.41
e5-mistral-7b-instruct 0.65 0.38 0.44 0.45 0.37 0.58 0.64
multilingual-e5-base 0.51 0.48 0.39 0.43 0.28 0.33 0.62
multilingual-e5-large 0.31 0.41 0.38 0.44 0.28 0.32 0.63
multilingual-e5-small 0.39 0.40 0.38 0.43 0.21 0.33 0.61
udever-bloom-1b1 0.27 0.40 0.30 0.44 0.16 0.27 0.62
udever-bloom-560m 0.21 0.38 0.25 0.36 0.08 0.22 0.54
laser2 0.30 0.32 0.27 0.35 0.12 0.26 0.48
bge-m3-custom-fr 0.42 0.29 0.42 0.42 0.31 0.39 0.58
sentence_croissant_alpha_v0.2 0.32 0.56 0.44 0.45 0.33 0.38 0.62
sentence_croissant_alpha_v0.3 0.38 0.58 0.44 0.44 0.35 0.41 0.60
mistral-embed 0.40 0.48 0.43 0.45 0.35 0.49 0.62
LaBSE 0.38 0.46 0.35 0.42 0.25 0.32 0.55
all-MiniLM-L12-v2 0.32 0.43 0.29 0.34 0.25 0.32 0.46
all-MiniLM-L6-v2 0.41 0.35 0.28 0.37 0.23 0.32 0.52
distiluse-base-multilingual-cased-v2 0.33 0.54 0.35 0.40 0.22 0.35 0.56
multi-qa-MiniLM-L6-cos-v1 0.27 0.54 0.26 0.35 0.14 0.26 0.49
paraphrase-multilingual-MiniLM-L12-v2 0.34 0.37 0.37 0.40 0.30 0.42 0.56
paraphrase-multilingual-mpnet-base-v2 0.31 0.42 0.38 0.41 0.31 0.45 0.54
sentence-t5-base 0.36 0.62 0.30 0.41 0.22 0.36 0.58
sentence-t5-large 0.31 0.59 0.32 0.42 0.25 0.40 0.62
sentence-t5-xl 0.32 0.63 0.34 0.42 0.27 0.41 0.60
sentence-t5-xxl 0.38 0.61 0.35 0.42 0.30 0.44 0.61
text2vec-base-multilingual 0.33 0.39 0.30 0.36 0.21 0.33 0.49
text-embedding-3-large 0.40 0.53 0.46 0.46 0.37 0.54 0.62
text-embedding-3-small 0.55 0.45 0.46 0.46 0.36 0.51 0.61
text-embedding-ada-002 0.49 0.68 0.42 0.45 0.35 0.54 0.65
voyage-code-2 0.35 0.57 0.41 0.45 0.35 0.51 0.62
universal-sentence-encoder-multilingual-3 0.40 0.61 0.36 0.44 0.24 0.38 0.57
universal-sentence-encoder-multilingual-large-3 0.40 0.24 0.38 0.41 0.23 0.38 0.54
xlm-roberta-base 0.24 0.29 0.24 0.40 0.09 0.20 0.52
xlm-roberta-large 0.22 0.34 0.19 0.43 0.06 0.21 0.57
Table 13: Performance of each model for Clustering.
25