A Fine-grained Sentiment Analysis Method Based on Dependency Tree and Graph Attention Network

With the rapid development of the Internet, e-commerce and social media have continued to develop and grow. Merchants tend to better understand users’ attitudes and emotional tendencies through comments. As the number of user reviews increases, the previous sentiment analysis methods have highlighted the problems of high cost and high error rate. How to use more advanced methods to analyse comments has become an urgent problem to be solved. In order to solve the above problems, in this article, we propose a fine-grained sentiment analysis method based on dependency tree and graph neural network, which can help businesses and social network platforms to identify users’ sentiment tendencies and can be subsequently used in recommendation systems and public opinion analysis systems. The experimental results show that our scheme has achieved the best results.


Introduction
With the rapid development of the Internet, e-commerce continues to grow and develop, and many excellent companies have emerged, such as Taobao, Jingdong. With the development of big data technology and the increase in the number of reviews, merchants tend to better understand users' attitudes and emotional tendencies through reviews.
Traditional sentiment analysis methods based on document-level and sentence-level cannot accurately judge different aspects of sentiment in sentences. For example, in the sentence 'The menu is limited but the dishes are excellent.', the sentiment polarity is different for each aspect. Aspectbased sentiment analysis (ABSA) can judge the sentiment polarity of each aspect in a sentence, which is very suitable for product reviews.
Existing work on aspect-based sentiment analysis (ABSA) can be classified into two main aspects, those methods examine sentiment polarity using attention mechanisms in neural networks and those utilizing syntax structures. Attention mechanisms can be considered as an implicit method of using sentence structure, because opinion words usually seem not far away from all aspects. Such methods have made gratifying progress. For example, Wang et al. [1] introduced an attention-based LSTM model to identify those sentiment information related to a specific aspect. Tang et al. [2] proposed a multi-hop attention Memory Network model. Moreover, to obtain long-distance opinion words for aspects, Chen et al. [3] proposed a multi-layer attention neutral network. Fan et al. [4] introduced a multi-grained attention network with different granularities. With the introduction of the pre-trained language model BERT [5] , many good models have been proposed and achieved good performance. For example, Xu et al. [6] utilized an added corpus to train BERT and proved its effectiveness in ABSA. Sun et al. [7] transformed ABSA into a sentence-pair classification task by introducing some auxiliary sentences. Other works have attempted to include grammatical information directly in ABSA. Since aspects are often assumed to be the core of this task, it is important to establish syntactic links between each target aspect and other words. In order to identify the relationship between aspects and potential opinion words, Qiu et al. [8] manually defined some syntactic rules. And then Liu et al. [9] obtained Partially aligned links with these syntactic rules. Afterward, a large number of neural network-based models were applied in this work. In another work, Wang et al. [10] integrated RNN with conditional random fields(CRF) for the co-extraction of aspects and opinion terms. Moreover, He et al. [11] introduced an attention model that incorporates syntactic information into the attention mechanism, which can better capture the semantics of the viewpoint target.
Recently, graph neural networks have achieved good results in many NLP tasks. However, the biggest problem is how to represent the data as a grid-like structure. To learn node representations from a dependency tree, Zhang et al. [12] proposed a graph convolutional network(GCN) over the dependency tree and Sun et al. [13] presented a convolution over a dependency tree (CDT) model. For a similar purpose, Huang et al. [14] propose target-dependent graph attention network(TD-GAT) utilizing the dependency relationship between words.
In order to solve those problems, in this article, we propose a fine-grained sentiment analysis method based on dependency tree and graph neural network, which can help businesses and social network platforms to identify user sentiment tendencies, the experimental results show that our scheme has achieved the best results.

Our Scheme
In our scheme, there are two types of text that need to be processed, namely text and relation. The overall framework of our model is shown in Figure 1. It is mainly composed of two process: the prediction of text and the prediction of relation.

Data preprocessing
Before data processing, we first perform data preprocessing, the main work is to extract semantic relations. For an input sentence, we first use the biaffine parser [15] to extract the semantic relationship of the sentence. We represent the input as  However, utilizing an ordinary dependency tree can't express the sentiment polarity of different aspects. To solve this problem, we reshape the dependency tree based on aspect. The reshaped dependency tree is shown as Figure 3.

Data processing
In the process of data processing, there are two types of data that need to be processed, namely text and relation. For text, we use BERT for processing and get context-based prediction results For relation, we use the last hidden states of the pre-trained BERT for word representations and use relation-graph attention network (R-GAT) [16] to obtain the prediction results based on semantic relations.

{ }
where F is an attention function. In this paper, we use the scaled dot-product attention function [17] :

Model training
During model training, the data will be transmitted forward along the neural network, and the network will calculate the probability of sentiment polarity. The calculation formula is as follows: After applying R-GAT on an reshaped dependency tree, its output output i h is passed through a softmax layer and mapped to probabilities of sentiment polarities. Finally, we use the standard crossentropy loss as our objective function: ( , ) ( )= -log ( ) where A represents the aspects appearing in sentence S , D denotes all the sentence-aspects pairs and θ contains all the parameters.

Datasets and Implementation
In this paper, the datasets we used are from SemEval 2014 Task 4 [18] , the statistics of them can be found in Table 1. In this paper, we use 5 heads in our scheme, and the hidden size is set as 300. The dropout is set as 0.1, and set L2 regularization term -5 =10 λ . The learning rate -5 2 10 × is adopted for training.

Results
We compare our scheme with the baseline models, the experiment results are shown in Table 2. We can find that our model outperforms most of the baseline models and the performance can be significantly improved when considering relational heads in our reshaped dependency tree structure.
This demonstrates that our model is better at encoding the semantic relationship information. These results have demonstrated the effectiveness of our scheme in capturing important syntactic structures for sentiment analysis.

Conclusion
In this paper, we proposed a fine-grained sentiment analysis method utilizing Biaffine syntactic parser, dependency tree and graph attention network, which can help businesses and social network platforms to identify user sentiment tendencies and can be subsequently used in recommendation systems and public opinion analysis systems. The experimental results show that our scheme has achieved the best results.