Intrusion detection systems (IDSs) play an important role in identifying malicious attacks and threats in networking systems. As fundamental tools of IDSs, learning based classification methods have been widely employed. When it comes to detecting network intrusions in small sample sizes (e.g., emerging intrusions), the limited number and imbalanced proportion of training samples usually cause significant challenges in training supervised and semi-supervised classifiers. In this paper, we propose a general network intrusion detection framework to address the challenges of both and . The novelty of the proposed framework focuses on incorporating deep adversarial learning with statistical learning and exploiting learning based data augmentation. Given a small set of network intrusion samples, it first derives a Poisson-Gamma joint probabilistic generative model to generate synthesised intrusion data using Monte Carlo methods. Those synthesised data are then augmented by deep generative neural networks through adversarial learning. Finally, it adopts the augmented intrusion data to train supervised models for detecting network intrusions. Comprehensive experimental validations on KDD Cup 99 dataset show that the proposed framework outperforms the existing learning based IDSs in terms of improved accuracy, precision, recall, and F1-score.
@cite_26: 
@cite_22: Abstract With the growing usage of technology, intrusion detection became an emerging area of research. Intrusion Detection System (IDS) attempts to identify and notify the activities of users as normal (or) anomaly. IDS is a nonlinear and complicated problem and deals with network traffic data. Many IDS methods have been proposed and produce different levels of accuracy. This is why development of effective and robust Intrusion detection system is necessary. In this paper, we have built a model for intrusion detection system using random forest classifier. Random Forest (RF) is an ensemble classifier and performs well compared to other traditional classifiers for effective classification of attacks. To evaluate the performance of our model, we conducted experiments on NSL-KDD data set. Empirical result show that proposed model is efficient with low false alarm rate and high detection rate.
@cite_0: In this paper, we developed a combining classifier model based on tree-based algorithms for network intrusion detection. The NSL-KDD dataset, a much improved version of the original KDDCUP’99 dataset, was used to evaluate the performance of our detection algorithm. The task of our detection algorithm was to classify whether the incoming network traffics are normal or an attack, based on 41 features describing every pattern of network traffic. The detection accuracy of 89.24 was achieved using the combination of random tree and NBTree algorithms based on the sum rule scheme, outperforming the individual random tree algorithm. This result represents the highest result achieved so far using the complete NSL-KDD dataset. Therefore, combining classifier approach based on the sum rule scheme can yield better results than individual classifiers, giving us hope of better anomaly based intrusion detection systems in the future.
@cite_16: 
@cite_13: Countering cyber threats, especially attack detection, is a challenging area of research in the field of information assurance. Intruders use polymorphic mechanisms to masquerade the attack payload and evade the detection techniques. Many supervised and unsupervised learning approaches from the field of machine learning and pattern recognition have been used to increase the efficacy of intrusion detection systems (IDSs). Supervised learning approaches use only labeled samples to train a classifier, but obtaining sufficient labeled samples is cumbersome, and requires the efforts of domain experts. However, unlabeled samples can easily be obtained in many real world problems. Compared to supervised learning approaches, semi-supervised learning (SSL) addresses this issue by considering large amount of unlabeled samples together with the labeled samples to build a better classifier. This paper proposes a novel fuzziness based semi-supervised learning approach by utilizing unlabeled samples assisted with supervised learning algorithm to improve the classifier's performance for the IDSs. A single hidden layer feed-forward neural network (SLFN) is trained to output a fuzzy membership vector, and the sample categorization (low, mid, and high fuzziness categories) on unlabeled samples is performed using the fuzzy quantity. The classifier is retrained after incorporating each category separately into the original training set. The experimental results using this technique of intrusion detection on the NSL-KDD dataset show that unlabeled samples belonging to low and high fuzziness groups make major contributions to improve the classifier's performance compared to existing classifiers e.g., naive bayes, support vector machine, random forests, etc.
@cite_12: Redundant and irrelevant features in data have caused a long-term problem in network traffic classification. These features not only slow down the process of classification but also prevent a classifier from making accurate decisions, especially when coping with big data. In this paper, we propose a mutual information based algorithm that analytically selects the optimal feature for classification. This mutual information based feature selection algorithm can handle linearly and nonlinearly dependent data features. Its effectiveness is evaluated in the cases of network intrusion detection. An Intrusion Detection System (IDS), named Least Square Support Vector Machine based IDS (LSSVM-IDS), is built using the features selected by our proposed feature selection algorithm. The performance of LSSVM-IDS is evaluated using three intrusion detection evaluation datasets, namely KDD Cup 99, NSL-KDD and Kyoto 2006+ dataset. The evaluation results show that our feature selection algorithm contributes more critical features for LSSVM-IDS to achieve better accuracy and lower computational cost compared with the state-of-the-art methods.
Existing learning based IDSs utilize ML and DL models for distinguishing different types of network data. In the category of recent ML based IDSs, @cite_16 combined unsupervised clustering and supervised learning for robust network traffic classification. @cite_13 proposed semi-supervised fuzzy method for intrusion detection. Those methods establish NID models by learning knowledge from a large amount of unlabeled network data. Howbeit, the performance of their methods in detecting intrusions with small sample sizes remains unknown if abundant data are unavailable. Apart from those methods, supervised classification models such as LR, SVM, and random forest (RF) have been extensively applied for improving modern IDSs @cite_12 @cite_26 @cite_22 . To deal with the feature redundancy problem in training supervised classifiers, @cite_12 introduced mutual information based algorithm to select network features. @cite_26 adopted word embedding for extracting meaningful features of network data. In order to further alleviate the over-fitting problem, RF @cite_22 and tree algorithms @cite_0 have been employed to ensemble sub-classification models for robust NID. However, the data scarcity and data imbalance remain unsolved because the feature selection extraction and classifier aggregation do not increase the number of intrusion samples among different categories in the training set.