Bypassing Feature Squeezing by Increasing Adversary Strength
Yash Sharma Pin-Yu Chen



Diversity Regularized Spatiotemporal Attention
for Video-based Person Re-identification
    
Shuang Li^1Work done while employed at Disney Research Pitsburgh.Slawomir Bak^2[1] Peter Carr^2[1] Xiaogang Wang^1 
     ^1CUHK-SenseTime Joint Lab,
 The Chinese University of Hong Kong     ^2Argo AI {sli,xgwang}@ee.cuhk.edu.hk     {sbak,pcarr}@argo.ai


    Received date / Accepted date
===================================================================================================================================================================================================================================================================




Feature Squeezing is a recently proposed defense method which reduces the search space available to an adversary by coalescing samples that correspond
to many different feature vectors in the original space into a single
sample. It has been shown that feature squeezing defenses can be combined in a joint detection framework to achieve high detection rates against state-of-the-art attacks. However, we demonstrate on the MNIST and CIFAR-10 datasets that by increasing the adversary strength of said state-of-the-art attacks, one can bypass the detection framework with adversarial examples of minimal visual distortion. These results suggest for proposed defenses to validate against stronger attack configurations.	 




§ INTRODUCTION


Deep neural networks (DNNs) achieve state-of-the-art performance in various tasks in machine learning and artificial intelligence, such as image classification, speech recognition, machine translation and game-playing. Despite their effectiveness, recent studies have illustrated the vulnerability of DNNs to adversarial examples <cit.>. For instance, a carefully designed perturbation to an image can lead a well-trained DNN to misclassify. Even worse, effective adversarial examples can also be made virtually indistinguishable to human perception. Adversarial examples crafted to evade a specific model can even be used to mislead other models trained for the same task, exhibiting a property known as transferability <cit.>.

To address this problem, numerous defense mechanisms have been proposed; one which has achieved strong results is feature squeezing. Feature squeezing relies on applying input transformations to reduce the degrees of freedom
available to an adversary by “squeezing” out unnecessary input
features. The authors in <cit.> propose a detection method using such input transformations by relying on the intuition that if the original and
squeezed inputs produce substantially different outputs from the model, the input is likely to be adversarial. By comparing
the difference between predictions with a selected threshold
value, the system is designed to output the correct prediction for legitimate examples and reject adversarial inputs. By combining multiple squeezers in a joint detection framework, the authors claim that the system can can successfully detect adversarial examples from eleven state-of-the-art methods <cit.>.

In this paper, we show that by increasing the adversary strength of the state-of-the-art methods, the feature squeezing joint detection method can be readily bypassed. We demonstrate this on both the MNIST and CIFAR-10 datasets. We experiment with EAD <cit.>, a generalization of the state-of-the-art C&W L_2 attack <cit.>, and I-FGSM <cit.>, an iterative version of the often used fast gradient L_∞ attack <cit.>. For EAD, and C&W, we increase the adversary strength by increasing κ, which controls the necessary margin between the predicted probability of the target class and that of the rest. For I-FGSM, we increase the adversary strength by increasing ϵ, which controls the allowable L_∞ distortion. We find that adversarial examples with minimal visual distortion can be generated which bypass feature squeezing under these stronger attack configurations. Our results suggest that proposed defenses should validate against adversarial examples of maximal distortion, as long as the examples remain visually adversarial. 



§ EXPERIMENT SETUP







Two types of feature squeezing were focused on by the authors in <cit.>: (i) reducing the color bit depth of images; and (ii) using smoothing (both local and non-local) to reduce the variation among pixels. For the detection method, the model’s original prediction is compared with the prediction on the squeezed sample using the L_1 norm. As a defender typically does not know the exact attack method, multiple feature squeezers are combined by outputting the maximum distance. The threshold is selected targeting a false positive rate below 5% by choosing a threshold that is exceeded by no more than 5% of legitimate samples.

For MNIST, the joint detector consists of a 1-bit depth squeezer with 2x2 median smoothing. For CIFAR-10, the joint detector consists of a 5-bit depth squeezer with 2x2 median smoothing and a non-local means filter with a 13x13 search window, 3x3 patch size, and a Gaussian kernel bandwidth size of 2. We use the same thresholds as used in <cit.>. We generate adversarial examples using the EAD <cit.> and I-FGSM attacks <cit.>. 

EAD generalizes the state-of-the-art C&W L_2 attack <cit.> by performing elastic-net regularization, linearly combining the L_1 and L_2 penalty functions <cit.>. The hyperparameter β controls the trade-off between L_1 and L_2 minimization. We test EAD in both the general case and the special case where β is set to 0, which is equivalent to the C&W L_2 attack. For MNIST and CIFAR-10, β was set to 0.01 and 0.001, respectively. We tune κ, which is a confidence parameter that controls the necessary margin between the predicted probability of the target class and that of the rest, in our experiments. κ is increased starting from 10 on both datasets, which was the value used in the feature squeezing experiments <cit.>. Full detail on the implementation is provided in the supplementary material.

For L_∞ attacks, which we will consider, fast gradient methods (FGM) use the sign of the gradient ∇ J of the training loss J with respect to the input for crafting adversarial examples <cit.>. I-FGSM iteratively uses FGM with a finer distortion, followed by an ϵ-ball clipping <cit.>. We tune ϵ, which controls the allowable L_∞ distortion, in our experiments. ϵ is increased starting from 0.3 on MNIST and 0.008 on CIFAR-10, which were the values used for the feature squeezing experiments <cit.>. Full detail on the implementation is provided in the supplementary material.

We randomly sample 100 images from the MNIST and CIFAR-10 test sets. For each dataset, we use the same pre-trained state-of-the-art models as used in <cit.>. We generate adversarial examples in the non-targeted case, force network to misclassify, and in the targeted case, force network to misclassify to a target class t. As done in <cit.>, we try two different targets, the Next class (t = label + 1 mod # of classes) and the least-likely class (LL).



§ EXPERIMENT RESULTS




The generated adversarial examples are tested against the proposed MNIST and CIFAR-10 joint detection configurations. In Tables <ref> and <ref>, the results of tuning κ for C&W and EAD are provided, and are presented with the results for I-FGSM at the lowest ϵ value at which the highest attack success rate (ASR) was yielded, against the MNIST and CIFAR-10 joint detectors, respectively. In all cases, EAD outperforms the C&W L_2 attack, particularly at lower confidence levels, indicating the importance of minimizing the L_1 distortion for generating robust adversarial examples with minimal visual distortion. Specifically, we find that with enough strength, each attack is able to achieve near 100% ASR against the joint detectors. 

In Figure <ref>, non-targeted MNIST adversarial examples generated by EAD are shown at κ={10,20,30}. In Figure <ref>, non-targeted CIFAR-10 adversarial examples generated by EAD are shown at κ={10,30,50}. Adversarial examples generated in the least-likely targeted case are provided in the supplementary material, These figures indicate that adversarial examples generated by EAD at high κ, which bypass the joint feature squeezing detector, have minimal visual distortion. This holds true for adversarial examples generated by I-FGSM with high ϵ on CIFAR-10, but not on MNIST.   



§ CONCLUSION


Feature Squeezing is a recently proposed class of input transformations which when combined in a joint detection framework has been shown to achieve high detection rates against state-of-the-art attacks. We show on the MNIST and CIFAR-10 datasets that by increasing the adversary strength, by tuning the confidence κ and L_∞ constraint ϵ for EAD and I-FGSM, respectively, the proposed joint detection configuration can be bypassed with adversarial examples of minimal visual distortion. These results suggest for proposed defenses to validate against stronger attack configurations, using the maximal adversary strength where examples remain visually similar to the inputs. For future work, we aim to validate if other recently proposed defenses are robust to strong adversaries.




iclr2018_workshop





§ SUPPLEMENTARY MATERIAL




 §.§ Attack Details


The targeted attack formulations are discussed below, non-targeted attacks can be implemented in a similar fashion. We denote by _0  and  the original and adversarial examples, respectively, and denote by t the target class to attack.



  §.§.§ EAD


EAD generalizes the state-of-the-art C&W L_2 attack <cit.> by performing elastic-net regularization, linearly combining the L_1 and L_2 penalty functions <cit.>. The formulation is as follows:

    minimize_  c · f(,t)+ β-_0_1 +  - _0_2^2 
       subject to  ∈ [0,1]^p,

where f(x,t) is defined as:

    f(,t)=max{max_j ≠ t [Logit()]_j - [Logit()]_t, - κ},


By increasing β, one trades off L_2 minimization for L_1 minimization. When β is set to 0, EAD is equivalent to the C&W L_2 attack. By increasing κ, one increases the necessary margin between the predicted probability of the target class and that of the rest. Therefore, increasing κ improves adversary strength but compromises visual quality. 

We implement 9 binary search steps on the regularization parameter c (starting from 0.001)
and run I=1000 iterations for each step with the initial learning rate α_0=0.01. For finding successful adversarial examples, we use the ADAM optimizer for the C&W attack and implement the projected FISTA algorithm with the square-root decaying learning rate for EAD <cit.>.  



  §.§.§ I-FGSM



Fast gradient methods (FGM) use the gradient ∇ J of the training loss J with respect to _0 for crafting adversarial examples <cit.>. For L_∞ attacks, which we will consider,  is crafted by
 
    =_0 - ϵ· (∇ J (_0,t)),

 where ϵ specifies the L_∞ distortion between  and _0, and  (∇ J) takes the sign of the gradient.
 
I-FGSM iteratively uses FGM with a finer distortion, followed by an ϵ-ball clipping <cit.>. 10 steps are used, and the step-size was set to be ϵ/10, which has been shown to be an effective attack setting in <cit.>.



 §.§ EAD Adversarial Examples in the Targeted Case (Figures 3 and 4)




In Figure <ref>, least-likely targeted MNIST adversarial examples generated by EAD are shown at κ={10,20,30}. In Figure <ref>, least-likely targeted CIFAR-10 adversarial examples generated by EAD are shown at κ={10,30,50}. Distortion is more apparent in the targeted case, particularly in the least-likely targeted case, but the examples are still visually adversarial.





 §.§ I-FGSM Adversarial Examples in the Non-Targeted Case (Figures 5 and 6)




In Figure <ref>, non-targeted MNIST adversarial examples generated by I-FGSM are shown at ϵ = {0.3,0.4,0.5,0.6,0.7,0.8.0.9,1.0}. In Figure <ref>, non-targeted CIFAR-10 adversarial examples generated by I-FGSM are shown at ϵ={0.008,0.04,0.07,0.1,0.2,0.3,0.4,0.5}. CIFAR-10 adversarial examples at high ϵ have minimal visual distortion, however MNIST examples at high ϵ, which yield the optimal ASR, have clear distortion. 

