text: 4.2.1 Heads with linguistic functions 
  The "heterogeneous" attention pattern shown in 
  Figure 3 could potentially be linguistically inter- 
  pretable, and a number of studies focused on iden- 
  In 
  tifying the functions of self-attention heads. 
  particular, some BERT heads seem to specialize 
  in certain types of syntactic relations. Htut et al. 
  (2019) and Clark et al. (2019) report that there 
  are BERT heads that attended signiﬁcantly more 
  than a random baseline to words in certain syntac- 
  tic positions. The datasets and methods used in 
  these studies differ, but they both ﬁnd that there are 
  heads that attend to words in obj role more than 
  the positional baseline. The evidence for nsubj, 
  advmod, and amod varies between these two stud- 
  ies. The overall conclusion is also supported by 
  Voita et al. (2019b)’s study of the base Transformer 
  in machine translation context. Hoover et al. (2019) 
  hypothesize that even complex dependencies like 
  dobj are encoded by a combination of heads 
  rather than a single head, but this work is limited 
  to qualitative analysis. Zhao and Bethard (2020) 
  looked speciﬁcally for the heads encoding negation 
  scope. 