text: Using the overall benchmark results, our goal 
  will be to answer the following research questions: 
  Q1: Is a model outstanding on all tasks? 
  As we are trying to find out whether one embed- 
  ding model is statistically better than the others for 
  French, the objective will also be to analyze the 
  performance of the models by tasks to facilitate 
  model choice for specific applications. 
  Q2: Are there any links between the model charac- 
  teristics and performance? 
  In section 3.2, we undertook the substantial task of 
  gathering the characteristics of all evaluated mod- 
  els. The goal here will be to analyze their impact 
  on performance and draw conclusions about, for 
  example, the relationship between embedding di- 
  mension and model ranking on the benchmark. 
  Q3: Do monolingual models have multilingual ca- 
  pabilities? 
  We interrogate the ability of a model trained exclu- 
  sively in one language to perform well in another 
  language. 
  Q4: Are there any correlations between datasets 
  
  • Model types: There are several strategies to 
  generate text embeddings such as aggregat- 
  ing (e.g. with average pooling) token-level 
  embeddings from raw pre-trained models, or 
  adding an extra contrastive learning step on a 
  sentence similarity task with, optionally, ad- 
  ditional transformation layers. We included 
  models of all types in our benchmark, summa- 
  rizing the model type information under two 
  relevant criteria: finetuned vs pretrained, and 
  trained for sentence similarity or not. 