text: In a little over a year, BERT has become a ubiq- 
  uitous baseline in NLP experiments and inspired 
  numerous studies analyzing the model and propos- 
  ing various improvements. The stream of papers 
  seems to be accelerating rather than slowing down, 
  and we hope that this survey helps the community 
  to focus on the biggest unresolved questions. 
  
  9 Acknowledgements 
  
  We thank the anonymous reviewers for their valu- 
  able feedback. This work is funded in part by 
  the NSF award number IIS-1844740 to Anna 
  Rumshisky. 