text: 4 Localizing linguistic knowledge 
  
  4.1 BERT embeddings 
  
  In studies of BERT, the term "embedding" refers 
  to the output of a Transformer layer (typically, the 
  ﬁnal one). Both conventional static embeddings 
  (Mikolov et al., 2013) and BERT-style embeddings 
  can be viewed in terms of mutual information max- 
  imization (Kong et al., 2019), but the latter are 
  contextualized. Every token is represented by a 
  vector dependent on the particular context of occur- 
  rence, and contains at least some information about 
  that context (Miaschi and Dell’Orletta, 2020). 