title: Entity and Evidence Guided Relation Extraction for DocRED
authors: Kevin Huang Guangtao Wang Tengyu Ma Jing Huang
text: 
  Hope Amid of a Pandemic: 
  
  Is Psychological Distress Alleviating in South America while Coronavirus is still on Surge?
      Josimar E. Chire-Saire10000-0003-2942-0853 Khalid Mahmood 2
  
      December 30, 2023
  ======================================================================================================================
  
  
  
  
  
  Document-level relation extraction is a challenging task which requires reasoning over multiple sentences in order to predict relations in a document. 
  In this paper, we propose a joint training framework E2GRE (Entity and Evidence Guided Relation Extraction) for this task.
  
  
  
  
  
  
  First, we introduce entity-guided sequences as inputs to a pretrained language model (e.g. BERT, RoBERTa).
  These entity-guided sequences help a pretrained language model (LM) to focus on areas of the document related to the entity.
  
  Secondly, we guide the fine-tuning of the pretrained language model by using its internal attention probabilities as additional features for evidence prediction.
  
  
  Our new approach encourages the pretrained language model to focus on the entities and supporting/evidence sentences. 
  
  We evaluate our E2GRE approach on DocRED, a recently released large-scale dataset for relation extraction. 
  Our approach is able to achieve state-of-the-art results on the public leaderboard across all metrics, showing that our E2GRE is both effective and synergistic on relation extraction and evidence prediction.
  
  
  
  
  
  
  
  
  
  
  
  
  § INTRODUCTION
  
  
  
  
  Relation Extraction (RE), the problem of extracting relations between pairs of entities in plain text, has received increasing research attention in recent years <cit.>. It 
  has important downstream applications to many other Natural Language Processing (NLP) tasks, such as Knowledge Graph Construction <cit.>, Information Retrieval, Question Answering <cit.> and Dialogue Systems <cit.>. 
  
  
  
  
  
  The majority of existing RE datasets focus on predicting intra-sentence relations, i.e., extracting relations between entity pairs in the same sentence. For example, SemEval-2010 Task 8 <cit.>, and TACRED <cit.> are two popular RE datasets with intra-sentence relations.
  These datasets have facilitated much research progress in this area such as <cit.> on SemEval-2010 Task 8 and <cit.> on TACRED. 
  
  However, in real world applications, the majority of relations are expressed across sentences.
  
  
  Figure <ref> shows an example from the DocRED dataset <cit.>, which requires reasoning over three evidence sentences to predict the relational fact that “The Legend of Zelda", is the publisher of “Link". 
  
  In this paper, we focus on the document-level relation extraction problem and design a method to facilitate document-level reasoning.
  We work on the DocRED <cit.>, a recent large-scale document-level relation extraction dataset.
  This dataset is annotated with a set of named entities and relations, as well as a set of supporting/evidence sentences for each relation. 
  Over 40% of the relations in DocRED require reasoning over multiple sentences. And supporting/evidence sentences can be used to provide an auxiliary task for explainable relation extraction. 
  
  
  
  A natural attempt to solve this problem is to fine-tune the large pretrained Language Models (LMs) (e.g., GPT <cit.>, BERT <cit.>, XLnet <cit.>, RoBERTa <cit.>), a paradigm that has proven to be extremely successful for many NLP tasks. 
  For example, all recent papers on DocRED have used BERT as an encoder to obtain the state-of-the-art results <cit.>. 
  However, naively adapting pretrained LMs for document-level RE faces a key issue that limits its performance. Due to the length of a given document, there are more entities pairs with meaningful relations in document-level relation extraction than in the intra-sentence relation extraction.
  A pretrained LM has to simultaneously encode information regarding all pairs of entities for relation extraction.
  Therefore, attention values that the pretrained LM gives over all the tokens are more uniform for document-level RE compared to intra-sentence RE. 
  This problem of having more uniform attention values limits the model's ability to extract information from relevant tokens from the document, limiting the effectiveness of the pretrained LM. 
  
  
  
  In order to mitigate this problem, we propose our novel Entity and Evidence Guided Relation Extraction (E2GRE).
  For each entity in a document, we generate a new input sequence by appending the entity to the beginning of a document, and then feed it into the pretrained LM. 
  Thus, for each document with N_e entities, we generate N_e entity-guided input sequences for training.
  By introducing these new training inputs, we encourage the pretrained LM to focus on the entity that is appended to the start of the document. 
  We further exploit the pretrained LM by directly using internal attention probabilities as additional features for evidence prediction. 
  The joint training of relation extraction and evidence prediction helps the model locate the correct semantics that are required for relation extraction. 
  Both of these ideas take advantage of pretrained LMs in order to make full use of pretrained LMs for our task. 
  Our main contribution is to propose the E2GRE approach, which consists of the two main ingredients below:
  
  
  
  
  
  
      
    * For every document, we generate multiple new inputs to feed into a pretrained language model: we concatenate every entity with the document and feed it as an input sequence to the language model. This allows the fine-tuning of the internal representations from the pretrained LM to be guided by the entity. 
      
      
      
    * We further propose to use internal BERT attention probabilities as additional features for the evidence prediction.
      
      This allows the fine-tuning of the internal representations from the pretrained LM to be also guided by evidence/supporting sentences. 
      
      
      
  
  
  
  Each of these ideas give a significant boost in performance and by combining them, we are able to achieve the state-of-the-art results on DocRED leaderboard. 
  
  
  
  
  
  § RELATED WORK
  
  
  
   §.§ Relation Extraction
  
  Relation Extraction is a long standing problem in NLP that has garnered significant research attention. 
  Early work attempts to solve this problem used statistical methods with different types of feature engineering <cit.>. 
  Afterwards, neural models have shown better performance at capturing semantic relationship between entities.
  These methods include CNN-based approaches <cit.> and LSTM approaches <cit.>.
  
  
  
  On top of using CNNs/LSTM encoders, previous models add more layers to take advantage of these embeddings.
  
  For example, <cit.> introduced using hierarchical attentions in order to generate relational information from coarse-to-fine semantic ideas; <cit.> applied GCN over the pruned dependency trees, and <cit.> introduced Attention Guided Graph Convolutional Networks (AG-GCNs) over dependency trees. These models have shown good performance on intra-sentence relation extraction, however, some of them are not easily adapted for inter-sentence document-level RE.
  
  
  
  
  
  
  
  <cit.> were among the early work on cross sentences and document-level relation extraction.
  Most approaches for document-level RE are graph-based neural network methods.
  <cit.> first introduced a document graph being used for document-level RE;
  <cit.> proposed a graph-structured LSTM for cross-sentence n-ary relation extraction; and <cit.> further extended the approach to graph-state LSTM. 
  In <cit.>, an entity-centric, multi-scale representation learning on entity/sentence/document-level LSTM model was proposed for document-level n-ary RE task. <cit.> recently proposed a novel edge-oriented graph model that deviates from existing graph models.
  <cit.> proposed an induced latent graph to perform document-level relation extraction on DocRED.
  These graph models generally focus on constructing unique nodes and edges, 
  
  and have the advantage of connecting different granularity of information and aggregate them together.
  
  
  
  
  
  
  
   §.§ Pretrained Language Models
  
  
  Pretrained Language Models (LMs) are powerful tools which emerged in recent years. 
  Recent pretrained LMs <cit.> are Transformer-based <cit.>, and trained with enormous amounts of data. <cit.> was the first large pretrained transformer-based LM to be released, and immediately get the state-of-the-art performance on a number of NLP tasks. 
  New pretrained LM models such as XLNet <cit.> and RoBERTa <cit.> further increase the performance on the most NLP tasks. 
  
  In order to take advantage of the large amounts of text that these models have seen, we finetune all of the weights inside the model. 
  
  Finetuning on large pretrained LMs has been shown to be effective on relation extraction <cit.>. 
  Generally, large pretrained LMs are used to encode a sequence and then generate the representation of a head/tail entity pair to learn a classification <cit.>. 
  <cit.> introduced a new concept similar to BERT called “matching-the-black” and pretrained a Transformer-like model for relation learning. 
  The models were fine-tuned on SemEval-2010 Task 8 and TACRED achieved state-of-the-art results.
  Our method aims to improve the effectiveness of a pretrained LMs, and directly influence the finetuning of the pretrained LMs with our entity and evidence guided approach.
  
  
  
  
  § METHODS
  
  
  In this section, we introduce our E2GRE method. We first describe how to generate entity-guided inputs in Section <ref>. Then, we present the entity-guided RE (Relation Extraction) in Section <ref>. 
  Finally, we describe the entity and evidence-guided joint training for RE in Section <ref>.
  We use BERT as an embodiment of a pretrained LM, and use BERT when describing our methods.
  
  
  
  
  
  
   §.§ Entity-Guided Input Sequences
  
  
  
  
  The relation extraction task is to predict the relation between each pair of head entity and tail entity in a given document. 
  
  
  We design the entity-guided inputs to give BERT more guidance towards the entities when finetuning.
  Each training input is organized by concatenating the tokens of the first mention of a single entity, denoted by H (named Concatenated Head Entity), together with the document tokens D, to form: “[CLS]”+ H + “[SEP]” + D + “[SEP]”, which is then fed into BERT. 
  We generate such input sequences for each entity in the given document. 
  Therefore, for a document with N_e entities, N_e new entity-guided input sequences are generated and fed into BERT separately.
  
  
  
  
  
  Due to BERT's sequence length constraint of 512 tokens, if the length of the training input is longer than 512, we make use of a sliding window approach over the document:
  
  
  
  
  
  we separate the input into multiple sequences. The first sequence is the original input sequence up to 512 tokens. The second sequence is the same as the first sequence, with an offset to the document, such that it can reach the end.
  
  This is shown as “[CLS]”+ H + “[SEP]” + D[offset:end] + “[SEP]”. We combine these two input sequences in our model by averaging the embeddings, and compute the BERT attention probabilities of the tokens twice in the model.
  
  
  
   §.§ Entity-Guided Relation Extraction
  
  
  
  
  
  
  
  
  
  
  
  For a given training input, we have one head entity, which corresponds with the concatenated entity H in the input, and N_e-1 different tail entities, which are located within the document D.
  Our method predicts N_e-1 different relations for each training input, corresponding to N_e-1 head/tail entity pairs. 
  
  After passing a training input through BERT, we extract out the head entity embedding and a set of tail entity embeddings from the BERT output.
  
  We average the embeddings over the concatenated head entity tokens to obtain the head entity embedding h.
  This is shown as the Head Extraction in Fig. <ref>. 
  In order to extract the k-th tail entity embedding t_k, we locate the indices of the tokens of k-th tail entity, average the output embeddings of BERT at these indices to get t_k (i.e., Tail Extraction in Fig. <ref>).
  
  After obtaining the head entity embedding h∈ℝ^d and all tail entity embeddings {t_k|t_k∈ℝ^d} in a entity-guided sequence, where 1≤ k ≤ N_e-1, we feed them into a bilinear layer with the sigmoid activation function to predict the probability of i-th relation between the head entity h and the k-th tail entity t_k, denoted by ŷ_ik, as follows
  
      ŷ_ik = δ(h^TW_it_k + b_i)
  
  where δ is the sigmoid function, W_i and b_i are the learnable parameters corresponding to i-th relation, where 1≤ i ≤ N_r, and N_r is the number of relations. 
  
  Finally, we finetune BERT with a multi-label cross-entropy loss as follow:
  
      L_RE   = -1/N_r1/N_e-1∑_k=1^N_e-1∑_i=1^N_r (y_iklog(ŷ_ik)
          +(1-y_ik)log(1-ŷ_ik))
  
  
  
  
  
  
  
  During inference, the goal of relation extraction is to predict a relation for each pair of head/tail entity within a document.
  For a given entity-guided input sequence of “[CLS]”+ entity + “[SEP]” + document + “[SEP]”, the output of our model is a set of N_e -1 relation predictions.
  
  We combine the predictions from every sequence generated from the same document and with different head entity, in order to obtain all relation predictions over the document. 
  
  
  
  
  
   §.§ Evidence Guided Relation Extraction
  
  
  
  
  
  
    §.§.§ Evidence Prediction
  
  
  
  
  
  Evidence/supporting sentences are the sentences containing important supporting facts for predicting the correct relationships between head and tail entities. Therefore, evidence prediction is a good auxiliary task to relation extraction and also provides explainability for the model. 
  
  The objective of evidence prediction is to predict whether a given sentence is evidence/supporting sentence for a given relation.
  Let N_s be the number of sentences in the document. We first obtain the sentence embedding s∈ℝ^N_S× d by averaging all the embeddings of the words in s (i.e., Sentence Extraction in Fig. <ref>). 
  These word embeddings are derived from the BERT output embeddings. 
  
  Let r_i∈ℝ^d be the relation embedding of i-th relation (1≤ i ≤ N_r}), which is initialized randomly and learnable in our model. 
  
  We employ a bilinear layer with sigmoid activation function to predict the probability of the j-th sentence s_j being a supporting sentence w.r.t. the given i-th relation r_i as follows.
  
  
      f^i_jk   = s_jW_i^rr_i + b_i^r
      ŷ^i_jk   = δ(f^i_jkW_o^r + b_o^r)
  
  
  
      
  
  
      
  
  
  
  where s_j represents the embedding of jth sentence, W_i^r/b_i^r and W_o^r/b_o^r are the learnable parameters w.r.t. i-th relation.
  
  
  We define the loss of evidence prediction under the given i-th relation as follows:
  
      L_Evi   = -1/N_t1/N_s∑_k=1^N_t∑_j=1^N_s (y^i_jklog(ŷ^i_jk)
         +(1-y^i_jk)log(1-ŷ^i_jk))
  
  
  where y_ik^j∈{0, 1}, and y_ik^j = 1  means that sentence j is an evidence for inferring i-th relation.
  
  It should be noted that in the training stage, we use the embedding of true relation in Eq. <ref>. In testing/inference stage, we use the embedding of the relation predicted by the relation extraction model in Section <ref>.
  
  
  
    §.§.§ Evidence-guided Finetuning with BERT Attention Probabilities
  
  
  
  
  
  Internal attention probabilities of BERT help locate the areas within a document where the BERT model focuses on. 
  
  Therefore, these probabilities can guide the language model to focus on relevant areas of the document for relation extraction (See the attention visualization in Section <ref>). 
  In fact, we find that the areas with higher attention values are usually come from the supporting sentences. Therefore, we believe these attention probabilities can be helpful for evidence prediction. 
  For each pair of head h and tail t_k, we make use of the attention probabilities extracted from the last l internal BERT layers for evidence prediction.
  
  
  
  Let Q∈ℝ^N_h× L × (d/N_h) be the query and K∈ℝ^N_h× L × (d/N_h) be the key of the multi-head self attention layer, N_h be the number of attention heads as described in <cit.>, L be the length of the input sequence (i.e., the length of entity-guided sequence defined in Section <ref>) and d being the embedding dimension.
  We first extract the output of multi-headed self attention (MHSA) A∈ℝ^N_h× L × L from a given layer in BERT as follows. 
  These extraction outputs are shown as “Attention Extractor” in Fig. <ref>. 
  
      = (QK^T/√(d/N_h)) 
         _i = (QW_i^Q, KW_i^K)
         A =(_i, ⋯ ,_n)
  
  
  For a given pair of head h and tail t_k, we extract the attention probabilities corresponding to head and tail tokens to help relation extraction. Specifically, we concatenate the MHSAs for the last l BERT layers extracted by Eq. <ref> to form an attention probability tensor as: Ã_̃k̃∈ℝ^l × N_h× L × L.
  
  Then, we calculate the attention probability representation of each sentence under a given head-tail entity pair as follows. 
  
      
    * We first apply maximum pooling layer along the attention head dimension (i.e., second dimension) over Ã_k. The max values are helpful to show where a specific attention head might be looking at. Afterwards we apply mean pooling over the last l layers. 
      
      We obtain Ã_s = 1/l∑_i=1^l(Ã_ki),  Ã_s∈ℝ^L × L from these two steps.
      
      
    * We then extract the attention probability tensor from the head and tail entity tokens according to the start and end positions of in the document.
      We average the attention probabilities over all the tokens for the head and tail embeddings to obtain Ã_sk∈ℝ^L.
  
      
    * Finally, we generate sentence representations from Ã_sk by averaging over the attentions of each token in a given sentence from the document to obtain a_sk∈ℝ^N_s
      
  
  
      
      
      
      
      
  
  
  
  
  Once we get the attention probabilities a_sk, we combine a_sk with the evidence prediction result ŷ^s_ik of sentence s from Eq. <ref> to form the new sentence representation and feed it into a bilinear layer with sigmoid for evidence sentence prediction as follows:
  
      ŷ^ia_k = δ(a_skW_i^af_k^i + b_i^a)
  
  where f_k^i is the vector of fused representation of sentence embeddings and relation embeddings for a given head/tail entity pair.
  
  Finally, we define the loss of evidence prediction under a given i-th relation based on attention probability representation as follows: 
  
      L_Evi^a   = -1/N_t1/N_s∑_k=1^N_t∑_j=1^N_s (y^ia_jklog(ŷ^ia_jk)
         +(1-y^ia_jk)log(1-ŷ^ia_jk))
  
  where ŷ^ia_jk is the j-th value of ŷ^ia_k computed by Eq. <ref>.
  
  
  
    §.§.§ Joint Training with Evidence Prediction
  
  We combine the relation extraction loss and attention probability guided evidence prediction loss as the final objective function for the joint training:
  
      Loss = L_RE+λ_1*L_Evi^a
  
  where λ_1 > 0 is the weight factor to make trade-offs between two losses, which is data dependent.
  
  
  
  § EXPERIMENTS
  
  We present the experimental results of our model E2GRE and compare with previously established baselines and published results, as well as the public leaderboard results on DocRED.
  
  
  
   §.§ Dataset
  
  
  DocRED <cit.> is a large document-level data set for the tasks of relation extraction and evidence sentence prediction. 
  It consists of 5053 documents, 132375 entities, and 56354 relations mined from Wikipedia articles. 
  For each (head, tail) entity pair, there are 97 different relation types as the candidates to predict. 
  The first relation type is an “NA” relation between two entities, and the rest of them corresponds to a WikiData relation name.
  Each of the head/tail pair that contain valid relations also include a set of supporting/evidence sentences. 
  
  
  We follow the same setting in <cit.> to split the data into Train/Validation/Test for model evaluation to make a fair comparison. The number of documents in Train/Validation/Test is 3000/1000/1000, respectively.
  
  The dataset is evaluated with the metrics of relation extraction RE F1, and evidence Evi F1.
  There are also instances where relational facts may occur in the validation and train set, and so we also evaluate on the Ign RE F1, which removes these relational facts.
  
  
  
  
  
  
  
  
  
   §.§ Experimental Setup
  
  
  hyper-parameter Setting. The configuration for the BERT-base model follows the setting in <cit.>. We set the learning rate as 1e-5, λ_1 as 1e-4, the hidden dimension of the relation vectors as 108, and extract internal attention probabilities from last three BERT layers. 
  
   
  We conduct most of our experiments by fine-tuning the BERT-base model. The implementation is based on the PyTorch <cit.> implementation of BERT[https://github.com/huggingface/pytorch-pretrained-BERT]. We run our model on a single V100 GPU for 60 epochs, resulting in approximately one day of training. The DocRED baseline and our E2GRE model have  115M parameters[We will release the code after paper review.]. 
  
  
  Baseline Methods. We compare our model with the following published models.
  
  
  1. Context Aware BiLSTM. <cit.> introduced the original baseline to DocRED in their paper. They used a context-aware BiLSTM (+ additional features such as entity type, coreference and distance) to encode the document. Head and tail entities are then extracted for relation extraction.
  
  2. BERT Two-Step. <cit.> introduced finetuning BERT in a two-step process, where the model first does predicts the NA relation, and then predicts the rest of the relations.[BERT Two-Step is an arxiv preprint].
  
  3. HIN. <cit.> introduced using a hierarchical inference network to help aggregate the information from entity to sentence and further to document-level in order to obtain semantic reasoning over an entire document.
  
  4. BERT+LSR. <cit.> introduced using an induced latent graph structure to help learning how the information should flow between entities and sentences within a document.
  
  
  
  
   §.§ Main Results
  
  As shown in Table <ref>, our method E2GRE is the current state-of-the-art model on the public leaderboard for DocRED. 
  
  Table <ref> compares our method with the baseline models. From Table <ref>, we observe that our E2GRE method is not only competitive to the previous best methods on the development set, but also holds the following advantages over previous models.
  
  
  
  
  
  
  
  Our E2GRE method is not only competitive to the previous best methods on the development set, but also holds the following advantages over previous models.
  
  
    * Our method is more intuitive and simpler in design compared to the HIN model and BERT+LSR model. In addition, our method provides interpretable relation extraction with supporting evidence prediction.
  
  
  
  
  
  
  
  
  
    * Our method is also better than all other models on the Ign RE F1 metric.
  
  
  This shows that our model does not memorize relational facts between entities, but rather examine relevant areas in the document to generate the correct relation extraction.
  
  
  
  Compared to the original BERT baseline, our training time is slightly longer, due to the multiple new entity-guided input sequences. We examined with the idea of generating new sequences based on each head and tail entity pair, but such a method would scale quadratically with the number of entities in the document. Using our entity-guided approach strikes a balance between performance and the training time.
  
  
  
  
  
   §.§ Ablation Study
  
  
  Analysis of Method Components
  Table <ref> shows the ablation study of our method on the effectiveness of entity-guided and evidence-guided training. The baseline here is the joint training model of relation extraction and evidence prediction with BERT-base.
  
  We see that the entity-guided BERT improves the over this baseline by 2.5%, and evidence-guided training further improve the method by 1.7%.
  This shows that both parts of our method are important to the overall E2GRE method.
  Our E2GRE method not only obtains improvement on the relation extraction F1, but it also obtains significant improvement on evidence prediction compared to this baseline. 
  This further shows that our evidence-guided finetuning method is effective.
  
  
  
  
  
  
  
  
  Analysis of Number of BERT Layers. 
  
  We also conduct experiments to analyze the impact of the number of BERT layers used for obtaining attention probability values, see the results in Table <ref>.
  From this table, we observe that using more layers is not necessarily better for relation extraction. 
  One possible reason may be that the BERT model encodes more syntactic information in the middle layers <cit.>.
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
   §.§ Attention Visualizations
  
  
  
  
  Fig. <ref> shows an example from the validation set of our model. In this example, the relation between “The Legend of Zelda" and “Link" relies on information across multiple sentences in the given document. 
  
  
  
  Fig. <ref> shows the attention heatmap of naively applying BERT for relation extraction. 
  This heatmap shows the attention of each word receives from `The Legend of Zelda” and “Link”.
  We observe that the model is able to locate the relevant areas of “Link” and “Legend of Zelda series”, but the attention values over the rest of the document are very small. Therefore, the model has trouble in extracting out information within the document to generate a correct relation prediction. 
   
  
  In contrast, Fig. <ref> shows that our E2GRE model highlights the evidence sentences, particularly in the areas where it finds relevant information. 
  Phrases related to “Link” and “The Legend of Zelda series” are assigned with the higher weights. Words (such as“protagonist” or “involves”) linking these phrases together are also highly weighted. Moreover, the scale of the attention probabilities for E2GRE is also much larger for E2GRE compared to the baseline.
  All of these phrases and bridging words are located within the evidence sentences, and make our model better at evidence prediction as well.
  
  
  
  
  
  § CONCLUSION
  
  
  In order to more effectively exploit pretrained LMs for document-level RE, we propose a new approach called E2GRE (Entity and Evidence Guided Relation Extraction). We first generate new entity-guided sequences to feed into a LM, focusing the model on the relevant areas in the document. Then we utilize the internal attentions extracted from the last l layers to help guide an LM to focus on relevant areas of the document. Our E2GRE method improves performance on both RE and evidence prediction on DocRED dataset, and achieves the state-of-the-art performance on the DocRED public leaderboard.
  
  
  For future work, we plan to incorporate our ideas on using attention-guided multi-task learning to other NLP tasks with evidence sentences. Combining our approach with graph-based models for NLP tasks is another interesting direction to explore.
  
  
  
                       
  acl_natbib
  