text: 3 What knowledge does BERT have? 
  
  A number of studies have looked at the knowledge 
  encoded in BERT weights. The popular approaches 
  include ﬁll-in-the-gap probes of MLM, analysis of 
  self-attention weights, and probing classiﬁers with 
  different BERT representations as inputs. 