Recently, many deep neural networks (DNN) based modulation classification schemes have been proposed in the literature. We have evaluated the robustness of two famous such modulation classifiers (based on the techniques of convolutional neural networks and long short term memory) against adversarial machine learning attacks in black-box settings. We have used Carlini & Wagner (C-W) attack for performing the adversarial attack. To the best of our knowledge, the robustness of these modulation classifiers has not been evaluated through C-W attack before. Our results clearly indicate that state-of-art deep machine learning-based modulation classifiers are not robust against adversarial attacks.
@cite_0: Deep learning (DL), despite its enormous success in many computer vision and language processing applications, is exceedingly vulnerable to adversarial attacks. We consider the use of DL for radio signal (modulation) classification tasks, and present practical methods for the crafting of white-box and universal black-box adversarial attacks in that application. We show that these attacks can considerably reduce the classification performance, with extremely small perturbations of the input. In particular, these attacks are significantly more powerful than classical jamming attacks, which raises significant security and robustness concerns in the use of DL-based algorithms for the wireless physical layer.
@cite_5: While research on adversarial examples in machine learning for images has been prolific, similar attacks on deep learning (DL) for radio frequency (RF) signals and their mitigation strategies are scarcely addressed in the published work, with only one recent publication in the RF domain [1]. RF adversarial examples (AdExs) can cause drastic, targeted misclassification results mostly in spectrum sensing survey applications (e.g. BPSK mistaken for 8-PSK) with minimal waveform perturbation. It is not clear if the RF AdExs maintain their effects in the physical world, i.e., when AdExs are delivered over-the-air (OTA). Our research on deep learning AdExs and proposed defense mechanisms are RF-centric, and incorporate physical world, OTA effects. We here present defense mechanisms based on statistical tests. One test to detect AdExs utilizes Peak-to- Average-Power-Ratio (PAPR) of the DL data points delivered OTA, while another statistical test uses the Softmax outputs of the DL classifier, which corresponds to the probabilities the classifier assigns to each of the trained classes. The former test leverages the RF nature of the data, and the latter is universally applicable to AdExs regardless of their origin. Both solutions are shown as viable mitigation methods to subvert adversarial attacks against communications and radar sensing systems.
@cite_2: Several machine learning models, including neural networks, consistently misclassify adversarial examples---inputs formed by applying small but intentionally worst-case perturbations to examples from the dataset, such that the perturbed input results in the model outputting an incorrect answer with high confidence. Early attempts at explaining this phenomenon focused on nonlinearity and overfitting. We argue instead that the primary cause of neural networks' vulnerability to adversarial perturbation is their linear nature. This explanation is supported by new quantitative results while giving the first explanation of the most intriguing fact about them: their generalization across architectures and training sets. Moreover, this view yields a simple and fast method of generating adversarial examples. Using this approach to provide examples for adversarial training, we reduce the test set error of a maxout network on the MNIST dataset.
There does not exist much literature on adversarial attacks on modulation classification. Recently, @cite_0 used a variant of fast gradient sign method (FGSM) attack @cite_2 on modulation classification on CNN-based modulation classification to highlight the threat of the adversarial examples. FGSM is an adversarial sample crafting algorithm where the adversarial perturbation is calculated by taking a gradient step in the direction of the sign of the gradient of test example. @cite_5 also crafted the adversarial examples for modulation classification by using the FGSM perturbation generation algorithm. Most of the available results on the application of the adversarial attacks are reported by using the FGSM attack.