Feature graph construction with static features for malware detection
Binghui Zou Chunjie Cao Longjuan Wang Yinan Cheng Jingzhang Sun





1,2]Binghui Zou

1,2]Chunjie Cao

1,2]Longjuan Wang*

1,2]Yinan Cheng

1,2]Jingzhang Sun

Binghui Zou et al.


[1]School of Cyberspace Security, Hainan University, Haikou, China

[2]Key Laboratory of Internet Information Retrieval of Hainan Province, Hainan University, Haikou, China

Longjuan Wang, School of Cyberspace Security, Hainan University, Haikou 570228, China.
 wanglongjuan@hainanu.edu.cn

National Natural Science Foundation of China Enterprise Innovation and Development Joint Fund (No.U19B2044), the National Key Research and Development Program of China (No.2021YFB2700600), and the Natural Science Foundation of Hainan Province (No.621MS017).

[Abstract]Malware can greatly compromise the integrity and trustworthiness of information and is in a constant state of evolution. Existing feature fusion-based detection methods generally overlook the correlation between features. And mere concatenation of features will reduce the model's characterization ability, lead to low detection accuracy. Moreover, these methods are susceptible to concept drift and significant degradation of the model. To address those challenges, we introduce a feature graph-based malware detection method, MFGraph, to characterize applications by learning feature-to-feature relationships to achieve improved detection accuracy while mitigating the impact of concept drift. In MFGraph, we construct a feature graph using static features extracted from binary PE files, then apply a deep graph convolutional network to learn the representation of the feature graph. Finally, we employ the representation vectors obtained from the output of a three-layer perceptron to differentiate between benign and malicious software. We evaluated our method on the EMBER dataset, and the experimental results demonstrate that it achieves an AUC score of 0.98756 on the malware detection task, outperforming other baseline models. Furthermore, the AUC score of MFGraph decreases by only 5.884% in one year, indicating that it is the least affected by concept drift.







Feature graph construction with static features for malware detection
    [
    April 29, 2024
=====================================================================








§ INTRODUCTION


The Internet's rapid development has led to numerous network security concerns, with malware being one of the most significant threats in cyberspace. Malware has the potential to exfiltrate user data and compromise privacy, thereby inflicting substantial harm on individuals, businesses, and governments.  According to the 2020 Kaspersky Security Bulletin, approximately 10.18% of users with networked computers subjected at least one attack of malware during the preceding year. This statistic is accompanied by the alarming discovery of an average of 360,000 new malicious files detected daily, and a striking 21 ransomware families identified within the course of 2020 alone. The situation is further compounded by attackers who are motivated by financial gain and are actively diversifying their attack channels while simultaneously employing increasingly sophisticated techniques to evade detection. A research report by DeepInstinct <cit.>, a cybersecurity company, reveals that in 2020, there were hundreds of millions of cyberattack attempts on a daily basis. Furthermore, the report highlights a 358% overall increase in the number of malware programs compared to 2019, with ransomware alone increasing by 435%. According to the X-Force Threat Intelligence Index 2021 <cit.>, ransomware continues to pose the most significant threat to large organizations and enterprises. Conservative estimates suggest that ransomware authors earned profits of at least $123 million and stole approximately 21.6 TB of data in 2020. Researching effective and consistently detectable malware models is an urgent task in cybersecurity.

Traditional malware detection approaches focus on individual features of malware, such as semantic features obtained through static methods <cit.> or behavioral features obtained through dynamic approaches <cit.>. Dynamic analysis necessitates the simulation of the actual environment in which the application operates, which can be arduous and time-consuming. Conversely, static analysis renders a simpler and more efficient approach due to its lack of requirement for program execution. Whether employing static or dynamic analysis, the processing and selection of features can pose significant challenges when confronted with a multitude of features <cit.>. The quantity and significance of features can crucially impact detection accuracy, emphasizing the importance of considering both factors when developing detection methods. During feature selection, commonly utilized features include API calls, N-grams, DLLs, and opcodes, among others. While a carefully chosen combination of features may result in high detection efficiency, feature selection is not a straightforward task. While reducing the number of features can expedite the learning process and enhance detection efficiency, an insufficient number of features can lead to a decrease in detection accuracy. Furthermore, disregarding the interdependence among features and resorting to a simplistic concatenation of features can result in a significant loss of valuable information, thereby impeding the identification of potential relationships among features.

Recent research has shown that converting applications into grayscale or color images is a promising method for malware detection <cit.>. With the powerful feature extraction capability of Convolutional Neural Networks (CNNs), subtle differences in texture features on malicious images can be well identified, enabling malware detection and fine-grained classification between families. Although this approach does not require feature engineering, the number of bytes contained between benign software and different families of malware varies widely, which makes the difference in the size of the generated malicious images too large. Even though they can be resized to a uniform size, the images lose detailed information during the transformation process, which makes the detection and classification results unreliable. In addition, malware is constantly evolving, and existing models are often unable to achieve accurate classification in the face of unknown malware and are susceptible to conceptual drift, leading to model degradation. 

Concept drift refers to the unpredictable changes over time in the correspondence between input data and the target variable <cit.>. Studies have shown <cit.> that malware detection methods are susceptible to concept drift, resulting in a decline in performance, such as lower detection accuracy and F1 score values. Despite the existence of various approaches to address conceptual drift in other domains <cit.>, the majority of these methods rely on static data sources. Given the rapid proliferation of malware variants, these techniques are not well-suited for malware detection tasks. In malware detection tasks, existing approaches tackle the concept drift problem by periodically training the model <cit.> or through integration methods <cit.>. However, the cost of retraining is too high for the former, and the latter faces the challenge of difficult feature selection.

The emergence of graph neural network (GNN) has garnered significant research interest in recent years <cit.>, owing to its capacity for analyzing and characterizing graph-structured data. GNNs have demonstrated exceptional success in diverse domains, including but not limited to recommendation systems <cit.>, knowledge graphs <cit.>, and molecular structures <cit.>. GNN can define learnable composite functions directly on the graph, thereby extending classical networks (e.g., long short-term memory network, CNNs, and multilayer perceptron) to more irregular and general domains. While researchers have already utilized graph neural networks to malware detection <cit.> or classification tasks <cit.> with promising results, they have yet to consider the impact of concept drift.

To address these limitations, this research proposes MFGraph, a feature graph-based method for malware detection. First, the static features of binary PE files are extracted using LIEF <cit.>. Next, a graph structure is introduced to fuse the static features, which is modeled as a graph network to capture the relationships between different features. To learn robust representations of binary files from the constructed feature graphs, the graph embeddings are obtained by employing a deep graph convolutional network <cit.>, and the detection results are outputted by the classifier. Experiments conducted on the EMBER dataset containing 800K samples show that MFGraph outperforms the method of directly concatenating features in detection accuracy and can effectively reduce the effect of concept drift.

The main contributions of this study are as follows:


	
  * We propose a static feature fusion approach based on graph representation learning to construct a network of relationships between different features. The feature graph, composed of feature nodes, can provide a more comprehensive characterization of binary PE files, which can be utilized for malware detection tasks.
	
	
  * We demonstrate a robust feature extraction method that utilizes deep graph convolutional networks to get the constructed feature graphs and obtain distinguishing features between benign and malicious software.
	
	
  * We evaluate our proposed approach on the EMBER dataset, and the results demonstrate that MFGraph can learn the potential relationships between features and achieve the best detection accuracy compared to the baseline method of feature concatenation. Additionally, MFGraph exhibits the most stable performance and effectively mitigates the impact of concept drift.




The remainder of this paper is structured as follows: In Section <ref>, we provide a review of the relevant literature. Section <ref> introduces the MFGraph method and presents its details. In Section <ref>, we describe the experimental results and systematically evaluate the performance of MFGraph. We discuss the forms of feature graph, complexity and interpretability in Section <ref>. Finally, we present concluding remark and future work in Section <ref>.



§ RELATED WORK





 §.§ Sequence-based detection methods


Malware causes serious damage and threat to the security of computer devices in the Internet, which has caused widespread concern among anti-virus researchers. Traditional malware detection methods, such as signature-based <cit.> or behavior-based <cit.>, are inadequate in detecting malicious programs. The prerequisite for achieving a high detection accuracy is the presence of such signatures or behaviors in malware training samples, and attackers can easily evade detection through obfuscation techniques. To detect unknown malware, several signature generation algorithms have been proposed. For instance, DeepSign <cit.> is an automated malware signature generation and classification method, AndroSimilar <cit.> generates signatures to detect malicious programs by extracting statistically unlikely features, and MalGene <cit.> uses algorithms borrowed from bioinformatics to automatically locate system evasive behavior in call sequences, an automated technique for extracting and analyzing evasive features. However, these methods employ a relatively individual set of features that do not accurately characterize malware. Raff et al. <cit.> employed raw byte sequences of malware as input to a neural network, transforming the detection of raw bytes into a sequence problem that deals with over 2 million time steps. Nevertheless, the excessively long sequences come at the cost of a vast number of parameters and extreme difficulty in training.

With the advancement of Machine Learning (ML) techniques, systems utilizing ML for malware detection <cit.> have become increasingly prevalent. Many researchers are currently focusing on API-based dynamic feature extraction techniques <cit.>. Rabadi et al. <cit.> proposed a lightweight dynamic feature extraction technique that combines API parameters and utilizes it for malware detection and classification. However, dynamic analysis is time-consuming and requires extensive feature engineering <cit.>, whereas malware detection necessitates quick detection results. To investigate the impact of concept drift, Ma et al. <cit.> compared disassembly-based, binary-based, and image-based malware detection methods and found that these methods are susceptible to concept drift, as evidenced by a decrease in F1 score scores. Additionally, Alsulami et al. <cit.> investigated the loss of malware detection performance under the influence of concept drift. Hu et al. <cit.> proposed an integrated learning method ENBCS to handle the concept drift in the task of Android malware detection and achieved favorable outcomes, provided that the program's permissions, operations, and code are obtained. However, accessing the code layer's functions can be challenging.



 §.§ Image-based detection methods


CNNs have demonstrated powerful classification performance in malware detection task, which has sparked the attention of many security researchers. Nataraj et al. <cit.> first proposed to transform malware into grayscale images and classify them using ML-based methods, providing a new idea in the field of malware detection. Since then, numerous techniques for detecting image-based malware have been suggested, such as HIT4Mal<cit.>, SERLA<cit.>, DeepVisDroid<cit.>, MCFT-CNN <cit.>.  HIT4Mal is a noteworthy color coding design for malware images that leverages a combination of entropy coding and character class schemes on a Hilbert curve to achieve an accuracy of 93.01%. SERLA is a deep neural network architecture that integrates SEResNet50, Bi-LSTM, and Attention mechanisms, and transforms malware into a three-channel malicious image. DeepVisDroid is another hybrid deep learning model for grayscale malware image classification, while MCFT-CNN accomplishes the task of classifying malicious images through transfer learning. In addition, Vasan et al. <cit.> transformed binary files into color images and detected malware variants by a fine-tuned CNN structure. In the same year, their other work IMCEC used grayscale images as input and learned classification using a deep CNN network. Although image-based detection methods have had some success, however, these methods overlook the potential model degradation caused by concept drift that may occur within real-world environments.



 §.§ Graph structure-based detection methods


The primary objective of GNN is to acquire node-level or graph-level representations of input graphs, consequently offering a more generalized approach for data abstraction. HomDroid <cit.> detects covert malware by assessing the homogeneity of call graphs, exhibiting better detection accuracy than PerDroid, Drebin, and MaMaDroid. IntDroid <cit.> regards an application's function call graph as a complex social network and represents the graph's semantic features by computing the average closeness between sensitive API calls and the central node. To achieve Android malware detection, Hou et al. <cit.> implemented the first structured heterogeneous information networks. Fan et al. <cit.> developed the malware detection system Scorpion. However, since it is a node classification problem, the graph structure needs to be reorganized for node embedding when new nodes arrive. Yan et al. <cit.> utilized graph convolutional networks for malware classification, characterizing malware using control flow graphs and embedding structural information in control flow graphs using deep graph convolutional networks to achieve effective and efficient malware classification. However, this approach requires expensive program analysis, and they did not consider the impact of concept drift.



Compared to the previous work outlined in Table <ref>, the study presented in this paper differs in three primary aspects. 1) We directly extract static features of binary PE files using LIEF without complex feature selection. 2) We employ a graph structure to fuse static features to explore the potential relationships between features. 3) We examine the impact of concept drift on existing detection methods over an extended timeline.




§ PROPOSED METHOD





The workflow diagram of the malware detection method we propose is shown in Figure <ref>, and the Malware Feature Graph (MFGraph) model for malware detection comprises three essential modules: Feature Graph Modeling (FGM), Graph Representation Learning (GRL), and Classifier Module (CM). As illustrated in Figure <ref>, FGM employs LIEF <cit.> to extract static features from the raw binary PE file, models the program as a feature graph, and utilizes a graph structure to fuse multiple features to enhance nodes with structural features and propagation characteristics of the graph. This approach captures feature-to-feature correlations for a more comprehensive characterization of the application. Subsequently, we transform malware detection into a graph binary classification problem by utilizing a deep graph convolutional network for graph representation learning and a multilayer perceptron for classification learning. 

Given a program P, nine static features are obtained by the LIEF parser: General, Header, Imported, Exported, Section, Byte Histogram, Byte Entropy Histogram <cit.>, Data directories, and String. Further details of these features are presented in Table <ref>. FGM utilizes these nine static features to construct a graph G for program P and the core idea of constructing the feature graph is shown in Algorithm 1. In the case of obtaining G=(V,E), GRL generates the corresponding graph embedding E_G through deep graph convolution and outputs the classification result P_out via the CM, thereby enabling malware detection. During the malicious detection phase, the classifier outputs either 0 or 1, indicating benign and malicious, respectively.







 §.§ Feature graph construction


To construct an appropriate feature graph, we conducted a comprehensive analysis of nine static features and integrated numerous empirical experimental results to determine the final node connections illustrated in Figure <ref>, also known as the skeleton graph. We began by selecting String (F_Str) as the foundation for the feature graph due to its inclusion of global statistical data on strings found in executable files, including string count, average length, and occurrences of specific characters. Byte Histogram (F_BH) and Byte Entropy Histogram (F_BEH) offer insights into overall distribution and randomness respectively, complementing F_Str in a mutually reinforcing manner. Additionally, section (F_Sec) and data directories (F_D) provide details on code, data content, and locations of various data structures within files, these are specific string attributes directly linked to F_Str. Header (F_H) contains metadata pertaining to PE files, while it does not contain target strings itself, it offers crucial file structure information. Moreover, general properties (F_G) encompass fundamental characteristics of PE files such as size, type, import and export function count etc., closely associated with F_Str.



Using the methods mentioned above, we have connected nodes for seven features, leaving only imported (F_I) and exported (F_E) features pending. The import table in F_I lists functions that PE files need to call from external dynamic link libraries (DLLs), while the export table in F_E lists functions provided by PE files for other programs or DLLs to use. These will be transformed into specific string features. However, they are closely linked to F_G as it contains the count of imported and exported functions. Connecting F_I and F_E with F_Str would lead to information redundancy and reduce the depth of the feature graph, which is not ideal for node propagation. So far, we have successfully created a node graph for nine static features.




 §.§ Feature graph modeling


In malware detection task, it is often necessary to extract multiple features of an unknown program, followed by feature selection or feature fusion. Selecting valuable features becomes difficult with a large number of features. When processing features, a common method is feature fusion, which combines existing features or features from different levels, but simple concatenating of features does not handle the relationship between features well. Recently, feature fusion by graph representation has shown a very promising technique for feature fusion <cit.>, where various features are modeled as graph networks through graph structures, thus eliminating the need for tedious feature selection.



Following the idea of feature graph modeling, we form the extracted static features <cit.> into a graph network, as shown in the Figure <ref>, in which the edges represent the relationship between features, and the nodes represent the features of the program, which we call "feature nodes". Formally, each binary PE file can be represented as a graph G=(V,E), where V denotes a set of nodes divided into major nodes (Majornodes,V_major) and child nodes (Childnodes,V_child). Each major node V_major represents a static feature of a binary file. These features include: General (F_G), Header (F_H), Imported (F_I), Exported (F_E), Section (F_Sec), Byte Histogram (F_BH), Byte Entropy Histogram (F_BEH), Data directories (F_D), and String (F_Str). The child node V_child is a child feature of each V_major. F_I denotes feature Imported, f_i denotes the name of the program import function, and an application importing multiple functions is denoted as F_E={f_1,f_2,…,f_i}. Each graph can be represented with the help of the adjacency matrix A. Thus, we implement the feature graph modeling of the binary PE file.

The specific process of FGM can be described as follows: after obtaining the static features of the binary PE file, the feature format obtained is shown in Figure <ref>. We divide the features of malware into first-level features and second-level features, and first vectorize the former features. When vectorizing the first-level features, we first obtain the number of second-level features of the first-level features. If there are third-level features of the second-level features, then the second-level feature vector is composed of third-level features, and the first-level feature vector is composed of second-level features. F_I, the primary feature Imported, contains multiple second-level features f_dll_n. f_dll_n indicates the name of the exported DLL function. In addition, a significant amount of API calls will be made by the DLL, so the secondary feature f_dll_n also contains the tertiary feature f_api_i, which is shown as two nodes in the feature graph. F_G indicates general characteristics, including file size, resources, signatures and etc. As none of these secondary features have three-level features, the primary feature F_G, which consists of secondary features, is only shown as a node in the feature graph, and the node in the dashed box in Figure <ref> will not appear in the feature graph.







 §.§ Graph representation learning




  §.§.§ Graph convolution layer


Inspired by Zhang et al. <cit.>, we use a deep graph convolutional network for graph representation learning in a malware detection task. We denote the node attribute matrix and adjacency matrix of a graph G=(V,E) containing n vertices as X∈ R^n× c and A∈ Z^n× n, respectively. G is an undirected graph, where n is the number of vertices and each vertex is a c-dimensional feature vector. The process of graph convolution is given in the following equation:


    Z=f(D^-1AXW)


Where A=A+I denotes the adjacency matrix with the self-loop added to enable the propagation of the properties of the vertex itself back to itself. D denotes the corresponding diagonal matrix, W∈ R^c× c^' denotes the trainable parameter matrix, f denotes the nonlinear activation function, and Z ∈ R^n× c^' is the output matrix after activation.

The graph convolution operation propagates the attributes of each vertex to its neighbors. In this process, in order to aggregate the attributes at multiple scales, we perform multiple graph convolution operations and recursively perform the feature scale transformation as,


    Z^t+1=f(D^-1AX^tW^t)


Where Z^0=X, the t-th layer indicates that the input is Z^t∈ Z^n× c_t, and the c_t feature channels are mapped to c_t+1 by the parameter matrix W^t∈ R^c_t× c_t+1. Based on the connectivity of the structure, each vertex's newly acquired feature channel is propagated to its neighboring nodes. Furthermore, owing to the addition of the self-loop, the new feature channel is also propagated back to itself, which is then multiplied by the adjacency matrix A. This  in turn is normalized using D^-1, so that the vertices in the graph are able to search in a breadth-first manner in the graph to pass their own properties. The j-th feature channel of vertex i is treated as a linear combination of the j-th feature channel of all neighboring nodes of that vertex. After h graph convolution layers, the output Z_t of each layer is concatenated, yielding,


    Z^1:h=[Z^1,Z^2,……,Z^h]




  §.§.§ SortPooling layer





After multiple layers of convolution, the final output is n rows and ∑_1^hc_t columns of Z^1:h. Each row can be considered as a vertex "feature descriptor", which encodes the local substructure information of the node at multiple scales. Z^1:h is approximately equal to the finest continuous color in the kernel of the Weisfeiler-Lehman graph <cit.>, and then those are used to sort all vertices. Sorting is performed  in decreasing order according to the last channel of the last layer. If two vertices have the same value in the last channel, Z^h-1 of the last layer is used to continue sorting, and so on. In this way, it is possible to obtain a representation of the entire graph, which can then be trained using traditional neural networks. To improve model generalization and unify output, similar to the work <cit.>, a hyperparameter called pooling rate needs to be adjusted to prevent overfitting of the model by discarding some of the feature channels. The pooling rate is used to determine the number of channels k that retain the final output of Z^1:h. If n>k, the last n-k rows are deleted; if less than k, k-n rows  need to be filled so that the output has k rows, so that each graph can obtain a uniform feature output Z^sp.



  §.§.§ Remaining layer


In the original DGCNN <cit.>, a tensor Z^sp of size k×∑_1^hc_t is output after the sorting pooling layer, with each row representing a vertex and each column representing a feature channel, and finally the authors add a one-dimensional convolutional kernel of size k∑_1^hc_t× with a step size ∑_1^h c_t such that the output Z^sp of the sorting pooling layer ends up as a one-dimensional vector of size k × F, and F is the number of convolution kernels. This has been improved in the research <cit.>. The single-channel Conv1D layer is parametrically represented as W∈ R^1× k and the output E∈ R^1×∑_1^hc_t, which is implemented as shown in the following equation, where f is the nonlinear activation function. 


    E=f(W× Z^sp)


Based on the idea of graph embedding <cit.>, the output Z_i^sp of each row of the sorted pooling result is used as the embedding of the retained vertices, and then the weighted sum of the vertices, which is also the embedding E_G of the graph G.



 §.§ Classifier module


Building on earlier work <cit.>, we added a multilayer perceptron after the output of the graph convolution layer for malware detection task.
The multi-layer perceptron <cit.>, a deep learning classification model, comprises three distinct components: an input layer, an output layer, and a hidden layer, which may consist of multiple fully connected layers. In order to enhance network performance and avoid overfitting, we have incorporated a Dropout layer <cit.> subsequent to every fully connected layer within the hidden layer, as,


    H=σ(E_GW_h+b)


Where the weight parameters and deviation parameters of the hidden layer are denoted by W_h and b, respectively. σ denotes the activation function. We used softmax as the activation function to transform the output values of the classification into a probability distribution between [0,1].



§ EXPERIMENTS





 §.§ Experiment setup




  §.§.§ Dataset


The EMBER dataset, a considerably large benchmark dataset of malware, has been published by Endgame. EMBER comprises features that have been extracted from 1.1M binary samples, where 900K samples are allotted for training purposes, consisting of 300K malicious, benign, and untagged samples each. Moreover, there are an additional 200K samples reserved for testing, which includes 100K malicious as well as benign samples respectively. In order to analyze the impact of concept drift, we have systematically categorized 800K samples based on a timeline. Figure <ref> illustrates the distribution of samples from January to December 2018 in the EMBER dataset and provides specific values. As the samples prior to 2018 contained only malicious samples and no benign samples, they were not used for training and were therefore discarded.



  §.§.§ Evaluation metrics






In the malware detection task, as in Ref. <cit.>, we use AUC score, accuracy, and F1 score to evaluate the performance of various methods. The accuracy and F1 score are calculated as in the following equations:


    Accuracy=TP+TN/TP+TN+FP+FN



    F1-score=2× Precision × Recalll/Precision + Recall


Where Precision=TP/TP+FP, Recall=TP/TP+FN



 §.§ Hyper parameter selection




To search for the hyperparameters of MFGraph, we employed five-fold cross-validation. We used 80% of the data from January 2018 for training, which was divided equally into five copies. In each non-repeating cycle, we used four copies (80%) for training and the remaining one (20%) for validation. During the experiment, we trained each model for 20 epochs and recorded the F1 score of each epoch to determine the optimal model. MFGraph has four main hyperparameters to search, namely, pooling rate, number of layers, number of channels, and number of hidden neurons. We plotted the results of these hyperparameters in Figure <ref>, where the y-axis represents the F1 score, and the x-axis represents the search values of the hyperparameters.

Our findings demonstrate that MFGraph attains its highest F1 score when configured with three layers, 1024 neurons, 48 convolutional channels, and a pooling rate of 0.75 (K=28). While larger hyperparameters may consume more computational resources, they have little impact on F1 score improvement and may even result in degraded performance. As such, we have incorporated this optimal set of hyperparameters into the default parameters of MFGraph for malware detection tasks.










 §.§ Malware detection


The first concern we studied is the accuracy of MFGraph in detecting malware. According to the distribution of EMBER dataset, the training set was composed of 80% of all samples collected in January 2018, while the remaining 20% were utilized as the test set. Upon completing 20 epochs, the model converged and demonstrated a high level of performance, with an AUC score of 0.98756, an accuracy of 0.95026, and an F1 score of 0.95020. The accompanying ROC curve can be found in Figure <ref>. In addition, Table <ref> provides a comparison of the model's results to those of six other detection methods on the test set. MFGraph outperforms six other methods according to experimental results, particularly in terms of AUC score, MFGraph outperforms all of the baseline models by at least 3.708%, with differences ranging from 4.0911% to 31.629%. Specifically, the AUC score of MFGraph is 24.069% and 7.323% higher than that of LR-based, and DT-based models. Additionally, both accuracy and F1 scores of MFGraph are at least 3.513% and 3.539% higher than those of the other models. The results indicate that our proposed MFGraph is capable of performing well in malware detection even when dealing with an unbalanced distribution in the dataset. This experimental finding provides evidence of the effectiveness of MFGraph in malware detection, as it outperforms other models such as DT-based, SVM-based, KNN-based, AdaBoost-based, and MLP-based in terms of AUC score, accuracy, and F1 score.

FFDNN <cit.> is a neural network model with a deeper structure compared to MLP-based models, featuring more neurons and network layers. However, the number of neurons and layers does not necessarily correlate with improved performance. As shown in Table  <ref>'s experimental results, the shallower MLP model achieved higher accuracy, while FFDNN scored better on AUC and F1 metrics. There are notable performance differences in malware detection between SVM-based and KNN-based methods. Ref. <cit.> and Ref. <cit.> delve into these variances, emphasizing the impact of data preprocessing on performance. The latter method combines feature selection techniques to preserve maximum relevant information with minimal redundancy. Furthermore, Ref. <cit.> preprocesses PE file header information of malware using various methods and employs an SVM model for detection. While these approaches reduce some feature overheads, they still necessitate laborious feature selection processes. In contrast, MFGraph only requires extracting nine static features through the LIEF parser and organizing them into a feature graph. By utilizing a multilayer perceptron, it achieves precise detection while offering interpretability regarding connections among different features.

The superior performance of MFGraph over all other baselines can be attributed to its implementation of graph structures. This is because such structures encapsulate more semantic information and connections between the various feature nodes, leading to a more comprehensive understanding of the underlying knowledge. In contrast, many previous methods generally utilize direct concatenation of features without considering any intrinsic relationships between them, resulting in suboptimal malware characterization and diminished detection capabilities. Direct concatenation of features in MLP-based detection yields an accuracy  of only 0.68621, whereas the accuracy of learned representation vectors improves by 26.405% to 0.95026 when classifying them using MLP after first modeling the features with graph structures. Notably, our method outperforms all others in terms of AUC score, accuracy, and F1 score.



 §.§ Impact of concept drift




To evaluate the effect of concept drift, we employed the classifier that was trained on the January 2018 dataset to classify a new dataset consisting of data from February through December 2018. The dataset was partitioned into 12 subsets, with each subset representing a month of the year based on the time series. In Figure <ref>, we present the distribution of benign and malware in the EMBER dataset for each month of the year 2018, accompanied by their respective quantitative values. To predict data from February to December, we employed models trained on data collected during January 2018 and evaluated their performance using the AUC score, accuracy, and F1 score metrics. Table <ref> presents the AUC score for MFGraph in comparison to other methods, while Figure <ref> displays the accuracy trend plots over time for different methods. Additionally, Figure <ref> provides F1 scores for each method across 12 test subsets.

The experimental results indicate that previous detection methods, including LR-based, DT-based, SVM-based, KNN-based, AdaBoost-based, and MLP-based approaches, exhibit significant degradation in performance when applied to new data due to changes in data distribution or the emergence of new malware variants. This is particularly evident when classifiers trained on older datasets are utilized to classify new data. In the initial months, most detection methods demonstrate a relatively stable performance. However, their efficacy dwindles over time due to the presence of concept drift in the actual environment. Known data trained models face difficulty in predicting unknown malware variants because new variants often have some features added or removed. Existing methods typically concatenate features without considering their inter-relationships, which restricts access to global features of malware. Therefore, when concept drift occurs, models undergo degradation, leading to poor performance. The detection method proposed in this study, known as MFGraph, incorporates potential relationships between features by utilizing graph representation learning methods to perform feature embedding. This process results in a set of representation vectors possessing global properties, leading to enhanced stability in the detection performance. 

To comprehensively assess the impact of concept drift, we not only record the highest and lowest AUC score, accuracy, and F1 score achieved by each model tested on all 12 subsets in Table <ref> but also calculate the value of the change in each model's three metrics, specifically the maximum percentage point of decline. Additionally, the visual representation of this change is illustrated in Figure <ref>. The experimental results reveal that among the six models tested, KNN-based is the most susceptible to the impact of concept drift, with a significant AUC drop of 29.789% and corresponding drops in both accuracy and F1 score by as much as 46.193% and 46.853%, respectively, indicating the most notable degradation. In contrast, our proposed MFGraph demonstrates superior performance. Specifically, within the first six months, MFGraph maintains consistently high AUC, with a decrease of less than 4%, while other models, including LR-based, DT-based, SVM-based, KNN-based, AdaBoost-based, and MLP-based, experience decreases ranging from 6.575% to 24.368%.  In the evaluation of all tested subsets, our proposed MFGraph displays superior performance with the highest AUC score, accuracy, and F1 score. Furthermore, compared to the maximum value, MFGraph exhibits only a marginal decrease in the AUC score by 5.884%, indicating its robustness to changes in the dataset. In contrast, although SVM-based shows the least change in the AUC score among the other six models, it performs poorly overall, with a maximum AUC score of only 0.57845. Similarly, while LR-based has the least change in accuracy and F1 scores, its maximum values for these metrics are much lower than the performance achieved by MFGraph, with a maximum accuracy of 0.67603 and a maximum F1 score of 0.67322. The  results demonstrate that our proposed MFGraph is highly effective in mitigating the impact of concept drift, evident from its superior performance and stability compared to other existing malware detection methods.







§ DISCUSSION





 §.§ Construction forms of MFGraph





We outlined the step-by-step construction of the feature graph in Section <ref>. In graph representation learning, the broadcast mechanism enables non-adjacent nodes to access each other's information, creating numerous ways to link nine static features. To investigate alternative methods for constructing the feature graph, we conducted a series of experiments. Figure <ref>(2) illustrates that connecting F_D as an intermediary node between F_G and F_Sec without linking to F_Str resulted in a decrease in overall detection accuracy to 0.942, suggesting that F_D contains more relevant data directory information from F_Str. Removing the connection between F_D and F_Sec (Figure <ref>(3)) did not impact accuracy, indicating their linkage is not crucial. However, eliminating the connection between F_G and F_Str (Figure <ref>(4)) reduced accuracy to 0.941, revealing a weak relationship between F_D and F_G. Hence avoiding direct connections with performance representation may enhance capturing characteristics from both sides (F_D and F_Sec).

Due to the valuable information in F_D, including resource directory, relocation directory, debug directory, and loading configuration directory, these features can complement F_Str for a more comprehensive global representation performance. We initially linked F_D to F_H as illustrated in Figure <ref>(5). However, experimental results show that this linking method notably decreases detection performance. Consequently, we also connected F_D to F_G as shown in Figure <ref>(6). In contrast to Figure <ref>(3), where F_G is additionally connected with F_H and F_Sec but only achieves an accuracy of 0.939. Moreover, as depicted in Figure <ref>(7), we tried using F_H as an intermediary node by connecting it with both F_Str and F_G and associating it with F_Sec. Nevertheless, the accuracy was only 0.918. This highlights a direct correlation between F_G and F_Str, where utilizing F_H as an intermediary node for propagation is unsuitable. It is important to note that creating a circular structure by linking pairs of nodes (F_G, F_Sec, F_H, F_Str, F_BH, F_BEH, F_D) while each node is also connected with four other non-adjacent nodes resulted in the lowest accuracy shown in Figure <ref>(8). This confirms that certain features do not require direct connections. Otherwise, there will be significant redundant information during graph representation learning leading to a noticeable decline in malware detection performance.




 §.§ Complexity and interpretability


MFGraph utilizes graph structure to model malware's static characteristics, conducts graph representation learning via deep graph convolutional neural networks, and ultimately performs malware detection using a three-layer perceptron classifier. Deep graph convolutional networks are well-suited for handling data with graph structures, effectively capturing intricate dependencies between nodes. However, this increases the model's complexity. Table <ref> demonstrates that employing a perceptron-based approach directly to learn classification sample features results in a low accuracy of only 0.68 due to the loss of valuable information between features encoded as data. By initially encoding features using a graph structure and leveraging deep graph convolutional networks to learn potential relationships among features before classifying feature vectors rich in information through perceptron, the malware detection performance significantly improves to 0.95. This highlights that graph convolution offers better interpretability and can identify complex dependencies among different features, simple concatenation would lead to substantial information loss. Furthermore, Figure <ref> reveals that most static malware is associated with F_Str, the relationship between F_D and F_Sec is insignificant compared to that between F_D and F_Str which has a more significant impact on detection outcomes than those between F_D and F_G or F_D and F_Sec. Thanks to graph convolutional networks, MFGraph achieves superior performance while also providing better interpretability compared to alternative methods.

Regarding the three-layer perceptron classifier in MFGraph, it is evident that the more layers the network has, the higher the model complexity. Figure <ref> illustrates from experimental result that F1 scores for three and four-layer perceptron are nearly identical when transitioning from one to five layers. However, adding more layers can lead to decreased performance, possibly due to overfitting. Fewer layers may limit the network's non-linear expressive capacity and suboptimal detection performance. Consequently, we opted for a three-layer perceptron as MFGraph's final classifier. The experiments and analysis reveal a significant rise in network complexity after incorporating graph convolutional networks and three-layer perceptron in MFGraph. This enhancement has boosted accuracy by 27%, improving its ability to cope with concept drift while also enhancing interpretability somewhat—a trade-off we consider valuable.



§ CONCLUSION AND FUTURE WORK



In this research, a feature graph-based method MFGraph is proposed to detect malware. The binary PE files are conceptualized as a feature graph comprising distinct feature nodes, thereby facilitating the learning of correlations between these features. Subsequently, a deep graph convolutional network is employed to derive the embedding of the entire graph based on the constructed feature graph. Finally, the resulting learned representation vectors are inputted into a classifier composed of MLPs to generate the output results. According to the experimental results, MFGraph not only surpasses other linear concatenation-based baseline methods in AUC score, accuracy, and F1 score but also demonstrates its ability to effectively mitigate the effects of concept drift while exhibiting better stability.

Due to the limitation of the dataset, we are currently unable to evaluate the method's performance changes over extended periods in the presence of concept drift. Meanwhile, the constructed feature graph in this research does not consider in heterogeneity among feature nodes. In upcoming studies, we will conduct a thorough analysis of the impact of various feature types on characterization performance. Additionally, this paper does not evaluate the method's performance on the malware family classification task. Therefore, we plan to assess its ability to identify similar malware families in future studies.



§ AUTHORS CONTRIBUTIONS

Binghui Zou: Conceptualization; Formal analysis; Investigation; Methodology; Data curation; Writing – original draft. Chunjie Cao: Funding acquisition; Project administration; Supervision. Longjuan Wang: Funding acquisition; Project administration. Yinan Cheng: Validation; Visualization. Jingzhang Sun: Writing – review & editing.



§ ACKNOWLEDGMENTS

This research was supported by the National Natural Science Foundation of China Enterprise Innovation and Development Joint Fund (No.U19B2044), the National Key Research and Development Program of China (No.2021YFB2700600), and the Natural Science Foundation of Hainan Province (No.621MS017).



§ CONFLICTS OF INTEREST

The authors declare that they have no competing interests.



§ DATA AVAILABILITY STATEMENT

The data used to support the findings of this study have been deposited in the ember repository (https://github.com/elastic/ember).

vancouver


