text: A number of these models also have multilingual variants such 
  as mBERT [39] and mT5 [148] or are trained with some amount of 
  multilingual data such as GPT-3 where 7% of the training data was 
  not in English [25]. The performance of these multilingual mod- 
  els across languages is an active area of research. Wu and Drezde 
  [144] found that while mBERT does not perform equally well across 
  all 104 languages in its training data, it performed better at NER, 
  POS tagging, and dependency parsing than monolingual models 
  trained with comparable amounts of data for four low-resource 
  languages. Conversely, [95] surveyed monolingual BERT models 
  developed with more specific architecture considerations or addi- 
  tional monolingual data and found that they generally outperform 
  
  1https://www.microsoft.com/en-us/research/blog/turing-nlg-a-17-billion-parameter- 
  language-model-by-microsoft/ 