Compositional Semantic Parsing Across Graphbanks
Matthias Lindemann Jonas Groschwitz Alexander Koller

Raven: Open Surgical Robotic Platforms
    [
    Received: date / Revised version: date
==========================================


  bstract
  ^*Equal contribution


ntroduction

malgebra
anyCorpora

valuation

onclusion



acl_natbib





§ EDGE ATTACHMENT AND SOURCE HEURISTICS


This section presents details of the heuristics discussed in Section 3 of the main paper, concerning grouping (edge attachment) and source heuristics.

Tables <ref>, <ref>, <ref> and <ref> show our edge attachment and source assignment heuristics for DM, PAS, PSD and EDS respectively. The heuristics are broken down by the edge's label in the `Label' column (`*' is a wildcard matching any string). A checkmark () in the `To Origin' column means that all edges with this label are attached to their origin node, a cross () means the edge is attached to its target node. In EDS, all edges go with their origin in principle, but in order to improve decomposability, we attach an edge to its target if the target node has label udef_q or nominalization.

The graphbanks differ in the directionality of edges; in particular, modifier relations sometimes point from the head to the modifier (PSD, AMR) and sometimes from the modifier to the head (DM, PAS, EDS). Our edge assignment heuristics account for that, following the principles for grouping. In DM, for instance, we treat the BV edge (pointing from a determiner to its head noun) as a modifier edge, and thus, it belongs to the determiner, which happens to be at the origin of the edge.

The `Source' column specifies which source is assigned to an empty node attached to an as-graph, depending on the label of the edge with which the node is attached. If an as-graph has multiple attached edges with the same label (or labels that map to the same source), i.e. multiple nodes would obtain the same source, we disambiguate the sources by sorting the nodes with the same source in an arbitrary order and appending `2' to the source at the second node, `3' to the source at the third node and so on (the source at the first node remains unchanged).
In PSD, where this happens particularly often, we order the nodes with the same source in their word order rather than arbitrarily, to get more consistent AM dependency trees.
For example, if in PSD there are two nodes that are attached to an as-graph with CONJ.member edges (such as in Figure <ref>c in the main paper), the edge going to the left gets assigned an op source and the edge going to the right an op2 source.




  
Passive and object promotion. Following
<cit.>, we allow some source names to be changed or swapped in an as-graph constant after their original assignments. That is, after we build a constant according to the edge grouping and source assignments described above, we generate multiple variants of the constant that have different source names. We allow

    
  * object promotion, e.g. instead of an O3 source we may
      also use an O2 or O source, as long as they don't
      exist yet in the constant,
    
  * unaccusative subjects, i.e. an O source may become an S source if no S source is present yet in the constant, and
    
  * passive, i.e. switching O and S sources.

This allows more graphs to be decomposed, by allowing e.g. the coordination of a verb in active and a verb in passive, or the raising of unaccusative subjects. We also follow <cit.> in the following (quoted directly from their Section 4.2): “To make our as-graphs more consistent, we prefer constants that promote objects as far as possible, use unaccusative subjects, and no passive alternation, but still allow constants that do not satisfy these conditions if necessary.”






  
Reentrancy heuristics. We update the reentrancy patterns of <cit.> in the following way. No coordination node patterns are allowed in DM (since DM uses edges for coordination); Coordination nodes in PAS are characterized via their coord_ARG_i edges. In PSD and EDS, any node that has two arguments that themselves have a common argument can be a coordination node.

Raising in PAS is done with the coordination pattern; in the others, a node v where one argument w has an S source can be a raising node (that is, we add an [S]-annotation at the source that the v-constant has at node w), as long as the edge between v and w has label

    
  * ARG1 or ARG2 in DM,
    
  * PAT-arg in PSD, or
    
  * any label in EDS.

We use the same raising-style pattern for comparatives in PSD, where we use no condition on the source that is passed along, but the edge from v to w must have label CPR.









  
Randomized heuristics. The randomized heuristics we experimented with on the DM set choose edge grouping (to target or to origin) and source names for each edge label independently uniformly at random (but consistently across the corpus).


§ TRAINING AND PARSING DETAILS



We reimplemented the graph-based parser of <cit.> in AllenNLP. We deviate from the original implementation in the following:

	
  * We use a cross-entropy loss instead of a hinge loss on the edge label predictions.
	
  * We follow <cit.> in using the Chu-Liu-Edmonds algorithm instead of Eisner's algorithm.
	
  * We don't perform word dropout but regular dropout on the input.

We add a supertagger consisting of a separate BiLSTM, from whose states we predict delexicalized graph fragments and lexical labels with an MLP. Learned embeddings are shared between the BiLSTM of the supertagger and the dependency parser.

The hyperparameters are collected in table <ref>. We train the parser for 40 epochs and pick the model with the highest performance on the development set (measured in Smatch for EDS, not in EDM). We perform early stopping with patience of 10 epochs.
Every lemma (and word in the case of using GloVe embeddings) that occurs fewer than 7 times is treated as unknown.

We use BucketIterators (padding noise 0.1) and the methods implemented in AllenNLP for performing padding and masking.



  
Training with GloVe We use the 200-dimensional version of GloVe (6B.200d) along with 100-dimensional trainable embeddings. We use two layers in the BiLSTMs and train with a batch size of 48.



  
Training with BERT  When using BERT, we replace both the GloVe embeddings and the learned word embedding with BERT. Since BERT does not
provide embeddings for the artificial root of the dependency tree, we learn a separate embedding. In some graphbanks (DM, PAS, PSD), we also have an
artificial word at the end of each sentence, that is used to connect the graphs. From BERT's perspective, the artificial word is a period symbol.

When training with BERT, we use a batch size of 64 and only one layer in the BiLSTMs. We use the "large-uncased" model as available through AllenNLP and don't fine-tune BERT.





  
MTL In our Freda experiments, we have one LSTM per graphbank and one that is shared between the graphbanks. 
When we compute scores for a sentence, we run it through its graphbank-specific LSTM and the shared one. We concatenate the outputs and feed it to graphbank-specific MLPs.
Again, we have separate LSTM for the edge model (input to edge existence and edge label MLP) and the supertagging model. 
In effect, we have two LSTMs that are shared over the graphbanks: one for the edge model and one for the supertagging model.

All LSTMs have the hyperparameters detailed in table <ref>. In the case of UD, we don't use a graphbank-specific supertagger because there are no supertags for UD. We don't pool the UD treebanks together. 

In the MTL setup, we select the epoch with the highest development F-score for DM for evaluation on all test sets.


  
Parsing We follow <cit.> in predicting the best
unlabeled dependency tree with the Chu-Liu-Edmonds algorithm and then
run their fixed-tree decoder restricted to the 6 best supertags. This
computes the best well-typed AM dependency tree with the same shape as
the unlabeled tree. 

Parsing is usually relatively fast (between 30 seconds and 2 minutes for the test
corpora) but very slow for a few sentences very long sentences in the AMR test
corpora. Therefore, we set a timeout. If parsing with k supertags is
not completed within 30 minutes, we retry with k-1 supertags. If
k=0, we use a dummy graph with a single node. This happened 4 times over different runs on AMR with the basic version of the parser and once when using BERT.



  
Copy function In order to predict the lexical label for EDS and AMR, we predict only the difference to its lemma or word form. For instance, if the lexical label is "want-01", we try to predict  instead at the word in question, e.g. wanted, and restore the full form of the lexical label in postprocessing.




§ DETAILS OF PREPROCESSING AND POSTPROCESSING




  
DM, PAS and PSD We handle disconnected graphs with components that contain more than one node by adding an artificial word to the end of the sentence. We draw an edge from this word to one node in every weakly connected component of the graph. We select this node by invoking Stanford CoreNLP <cit.> to find the head of the span the component comprises. 

Disconnected components that only contain one word are treated as words without semantic contribution, which we attach to the artificial root (position 0) with an Ignore-edge.

Since the node labels in these graphbanks are the words of the sentences, we simply copy the words over to the graph.

We use the evaluation toolkit that was developed for the shared task: https://github.com/semantic-dependency-parsing/toolkithttps://github.com/semantic-dependency-parsing/toolkit.



  
EDS We only consider connected EDS graphs (98.5% of the corpus) and follow <cit.> regarding options for the tokenizer except for hyphenated words, which we split. Since EDS nodes are aligned with (character) spans in the sentence, we make use of this information in the decomposition. In our approach, however, we require every graph constant to stem from exactly one token. In order to enforce this, we assign nodes belonging to a multi-token span to an atomic span whose nodes are incident. For consistency, we perform this from left to right. We try to avoid creating graph constants that would require more than one root source. Where this fails, the graph cannot be decomposed.

We delete R-HNDL and L-HNDL edges only if this does not make the graph disconnected. Thus, we need heuristics for them (see table <ref>).

Before delexicalizing graphs constants, we need to identify lexical nodes. A node is considered lexical if has an incoming c-arg edge or if its label is similar to the aligned word, its lemma or its modified lemma. We compute the modified lemma by a few hand-written rules from the CoreNLP lemma. For instance, "Tuesday" is mapped to "Tue". We also re-inflect adverbs (as identified by the POS tagger) to their respective adjectives if possible, e.g. "interestingly" becomes "interesting". We perform this step in order to be able to represent the lexical label of more graph constants as function of the word which they belong to. The modified lemma is not used as input to the neural network.

When performing the delexicalization, we replace the character span information with placeholders indicating if this span is atomic (comprises a single word) or not. We restore the span information for every node with a very simple heuristic in postprocessing: If the span is atomic, we simply look up the character span in the original string. For nodes with complex spans, we compute the minimum of beginnings and the maximum of endings of its children. In terms of evaluation, the span information is relevant only for EDM. Comparing the graphs that we restore from our training data to the gold standard, we find that the upper bound is at 89.7 EDM F-score. The upper bound in terms of Smatch is at 96.9 F-score.

We use EDM in an implementation by <cit.>.



  
UD Since UD POS tags are different from the English PTB tagset, we use CoreNLP to tag the UD treebanks. We use the English treebanks EWT, GUM, ParTUT and LinES <cit.>.



  
AMR
We use the pre- and postprocessing pipeline of <cit.>. We conflate named entities in preprocessing. For instance, "New York" is conflated to one token "New_York". When such a graph constant is predicted, we restore the named entity prior to evaluation.





























