Dataset x Task Average # tokens# samples Reference LicenseAmazonReviewsClassification49.6 5000 McAuley and Leskovec (2013) N/AMasakhaNEWSClassification1398.2 422 Adelani et al. (2023) AFL-3.0MassiveIntentClassification11.4 2974 FitzGerald et al. (2023) N/AMassiveScenarioClassification11.4 2974 FitzGerald et al. (2023) N/AMTOPDomainClassification12.5 3193 Li et al. (2021) N/AMTOPIntentClassification 12.5 3193 Li et al. (2021) N/AAlloProfClusteringP2P 1021.8 2556 Lefebvre-Brossard et al. (2023) MITAlloProfClusteringS2S 8.8 2556 Lefebvre-Brossard et al. (2023) MITHALClusteringS2S 25.6 26233 Introduced by our paper Apache-2.0MasakhaNEWSClusteringP2P1398.1 422 Adelani et al. (2023) AFL-3.0MasakhaNEWSClusteringS2S21.7 422 Adelani et al. (2023) AFL-3.0MLSUMClusteringP2P 1062.1 15828 Scialom et al. (2020) OtherMLSUMClusteringS2S 20.8 15828 Scialom et al. (2020) OtherOpusparcusPC 9.7 1007 Creutz (2018) CC-BY-NC-4.0PawsX 34.9 2000 Yang et al. (2019) OtherSTSBenchmarkMultilingualSTS18.4 1379 May (2021) N/ASTS22 722.1 104 Chen et al. (2022) N/ASICKFr 15.1 4906 https://huggingface.co/datasets/Lajavaness/SICK-frApache-2.0DiaBLaBitextMining 12.02 5748 Bawden et al. (2021) CC-BY-SA-4.0FloresBitextMining 33.42 1012 Goyal et al. (2021) CC-BY-SA-4.0AlloprofReranking 48.3 - 1179.4 - 1196.42316 - 2975 - 22064 Lefebvre-Brossard et al. (2023) MITSyntecReranking 19.2 - 402.2 - 467.2100 - 100 - 917 Introduced by our paper Apache-2.0AlloprofRetrieval 48.31 - 1117.912316 - 2556 Lefebvre-Brossard et al. (2023) MITBSARDRetrieval 144.03 - 24530.8222 - 22600 Louis and Spanakis (2022) CC-BY-NC-SA-4.0SyntecRetrieval 19.22 - 295.65 100 - 90 Introduced by our paper Apache-2.0SummEvalFr 657.08 - 71.18 - 107.56100 - 1100 - 1600 Created from Fabbri et al. (2021) MIT
Table 3: Details of the data used for each task. The average number of tokens of texts is computed using the
cl100k_base tokenizer. For Reranking, the three numbers refer to the queries, the pairs of queries with relevant
documents and the pairs of queries with irrelevant ones, respectively. The pairs of queries and documents are
obtained from the 90 dataset’s documents. For Retrieval datasets, the two numbers refer to the queries and the
documents, respectively. For SummEvalFr, the three numbers refer to the texts, human and machine summaries.
References to all the datasets used are available.
AmazonReviewsClassification
MasakhaNEWSClassification
MassiveIntentClassification
MassiveScenarioClassification
MTOPDomainClassification
MTOPIntentClassification
AlloProfClusteringP2P
AlloProfClusteringS2S
HALClusteringS2S
MasakhaNEWSClusteringP2P
MasakhaNEWSClusteringS2S
MLSUMClusteringP2P
MLSUMClusteringS2S
AlloprofRetrieval
BSARDRetrieval
SyntecRetrieval
OpusparcusPC
PawsX
AlloprofReranking
SyntecReranking
SICKFr
STS22
STSBenchmarkMultilingualSTS
SummEvalFr
AmazonReviewsClassification
MasakhaNEWSClassification
MassiveIntentClassification
MassiveScenarioClassification
MTOPDomainClassification
MTOPIntentClassification
AlloProfClusteringP2P
AlloProfClusteringS2S
HALClusteringS2S
MasakhaNEWSClusteringP2P
MasakhaNEWSClusteringS2S
MLSUMClusteringP2P
MLSUMClusteringS2S
AlloprofRetrieval
BSARDRetrieval
SyntecRetrieval
OpusparcusPC
PawsX
AlloprofReranking
SyntecReranking
SICKFr
STS22
STSBenchmarkMultilingualSTS
SummEvalFr
0.95
0.96 0.95
0.97 0.95 1
0.96 0.95 0.99 0.99
0.96 0.94 0.99 0.99 1
0.96 0.97 0.95 0.95 0.94 0.94
0.96 0.94 0.98 0.98 0.97 0.97 0.95
0.96 0.95 0.97 0.97 0.97 0.96 0.96 0.98
0.95 1 0.95 0.95 0.94 0.94 0.97 0.94 0.95
0.95 0.97 0.97 0.97 0.97 0.96 0.95 0.97 0.97 0.97
0.95 0.98 0.94 0.95 0.94 0.94 0.97 0.94 0.95 0.98 0.96
0.95 0.98 0.94 0.94 0.94 0.94 0.97 0.94 0.95 0.98 0.96 1
0.96 0.97 0.95 0.95 0.95 0.95 1 0.96 0.96 0.97 0.95 0.96 0.97
0.95 0.95 0.94 0.95 0.94 0.94 0.96 0.95 0.96 0.95 0.95 0.94 0.94 0.96
0.93 0.94 0.93 0.93 0.93 0.92 0.95 0.93 0.94 0.94 0.93 0.94 0.94 0.95 0.97
0.96 0.93 0.97 0.97 0.97 0.97 0.94 0.97 0.96 0.93 0.96 0.93 0.93 0.94 0.94 0.92