text: improve BERT’s architecture, pre-training and ﬁne- 
  tuning. We conclude by discussing the issue of 
  overparameterization, the approaches to compress- 
  ing BERT, and the nascent area of pruning as a 
  model analysis technique. 