3 What knowledge does BERT have?
A number of studies have looked at the knowledge
encoded in BERT weights. The popular approaches
include ﬁll-in-the-gap probes of MLM, analysis of
self-attention weights, and probing classiﬁers with
different BERT representations as inputs.
3.1 Syntactic knowledge
Lin et al. (2019) showed that BERT represen-
tations are hierarchical rather than linear , i.e.
there is something akin to syntactic tree structure
in addition to the word order information. Ten-
ney et al. (2019b) and Liu et al. (2019a) also
showed that BERT embeddings encode informa-
tion about parts of speech, syntactic chunks
and roles. Enough syntactic information seems
to be captured in the token embeddings themselves
to recover syntactic trees (Vilares et al., 2020; Kim
et al., 2020; Rosa and Mare ˇcek, 2019), although
probing classiﬁers could not recover the labels of
distant parent nodes in the syntactic tree (Liu et al.,
2019a). Warstadt and Bowman (2020) report evi-
dence of hierarchical structure in three out of four
probing tasks.
As far as how syntax is represented, it seems
that syntactic structure is not directly encoded
in self-attention weights. Htut et al. (2019) were
unable to extract full parse trees from BERT heads
even with the gold annotations for the root. Jawahar
et al. (2019) include a brief illustration of a depen-
dency tree extracted directly from self-attention
weights, but provide no quantitative evaluation.
However, syntactic information can be recov-
ered from BERT token representations. Hewitt
and Manning (2019) were able to learn transfor-
mation matrices that successfully recovered syn-
tactic dependencies in PennTreebank data from
BERT’s token embeddings (see also Manning et al.,
2020). Jawahar et al. (2019) experimented with
transformations of the [CLS] token using Tensor
Product Decomposition Networks (McCoy et al.,
2019a), concluding that dependency trees are the
best match among 5 decomposition schemes (al-
though the reported MSE differences are very
small). Miaschi and Dell’Orletta (2020) performs
a range of syntactic probing experiments with con-
catenated token representations as input.
Note that all these approaches look for the
evidence of gold-standard linguistic structures,
and add some amount of extra knowledge to the
probe. Most recently, Wu et al. (2020) proposed a
4168
[CLS]Forthosewhofollowsocialmedia
transitions
on
Capitol
Hill , this will be alittle
different
.
[CLS]
For
those
who
follow
social
media
transitions
on
Capitol
Hill
,
this
will
be
a
little
different
.
0
1
2
3
4
5
Figure 1: Heatmap of the impact matrix for the sen-
tence “For those who follow social media transitions
on Capitol Hill, this will be a little different.”
3 Visualization with Impact Maps
Before we discuss speciﬁc syntactic phenomena,
let us ﬁrst analyze some example impact matri-
ces derived from sample sentences. We visual-
ize an impact matrix of a sentence by displaying
a heatmap. We use the term “impact map” to refer
to a heatmap of an impact matrix.
Setup. We extract impact matrices by feed-
ing BERT with 1,000 sentences from the English
Parallel Universal Dependencies (PUD) treebank
of the CoNLL 2017 Shared Task ( Zeman et al. ,
2017). We follow the setup and pre-processing
steps employed in pre-training BERT. An example
impact map is shown in Figure 1.
Dependency. We notice that the impact map
contains many stripes, which are short series of
vertical/horizontal cells, typically located along
the diagonal. Take the word “ different” as an ex-
ample (which is illustrated by the second-to-last
column in the impact matrix). We observe a clear
vertical stripe above the main diagonal. The inter-
pretation is that this particular occurrence of the
word “different” strongly affects the occurrences
of those words before it. These strong inﬂuences
are shown by the darker-colored pixels seen in the
second last column of the impact map. This ob-
servation agrees with the ground-truth dependency