text: However, syntactic information can be recov- 
  ered from BERT token representations. Hewitt 
  and Manning (2019) were able to learn transfor- 
  mation matrices that successfully recovered syn- 
  tactic dependencies in PennTreebank data from 
  BERT’s token embeddings (see also Manning et al., 
  2020). Jawahar et al. (2019) experimented with 
  transformations of the [CLS] token using Tensor 
  Product Decomposition Networks (McCoy et al., 
  2019a), concluding that dependency trees are the 
  best match among 5 decomposition schemes (al- 
  though the reported MSE differences are very 
  small). Miaschi and Dell’Orletta (2020) performs 
  a range of syntactic probing experiments with con- 
  catenated token representations as input. 
  
  Note that all these approaches look for the 
  evidence of gold-standard linguistic structures, 
  and add some amount of extra knowledge to the 
  probe. Most recently, Wu et al. (2020) proposed a 