Obfuscated Gradients Give a False Sense of Security:
Circumventing Defenses to Adversarial Examples

Anish Athalye * 1 Nicholas Carlini * 2 David Wagner 2

arXiv:1802.00420v2 [cs.LG] 15 Feb 2018

Abstract
We identify obfuscated gradients, a kind of gradient masking, as a phenomenon that leads to a false
sense of security in defenses against adversarial
examples. While defenses that cause obfuscated
gradients appear to defeat iterative optimizationbased attacks, we find defenses relying on this
effect can be circumvented. For each of the three
types of obfuscated gradients we discover, we
describe characteristic behaviors of defenses exhibiting this effect and develop attack techniques
to overcome it. In a case study, examining noncertified white-box-secure defenses at ICLR 2018,
we find obfuscated gradients are a common occurrence, with 7 of 8 defenses relying on obfuscated
gradients. Our new attacks successfully circumvent 6 completely and 1 partially.

1. Introduction
In response to the susceptibility of neural networks to adversarial examples (Szegedy et al., 2013; Biggio et al., 2013),
there has been significant interest recently in constructing defenses to increase the robustness of neural networks. While
progress has been made in understanding and defending
against adversarial examples in the white-box setting, where
the adversary has complete access to the network, a complete solution has not yet been found.
As benchmarking against iterative optimization-based attacks (e.g., (Kurakin et al., 2016a; Madry et al., 2018; Carlini & Wagner, 2017c)) has become standard practice in
evaluating defenses, new defenses have arisen that appear to
be robust against these powerful optimization-based attacks.
We identify one common reason why many defenses provide
apparent robustness against iterative optimization attacks:
obfuscated gradients, a term we define as a special case of
gradient masking (Papernot et al., 2017). Without a good
gradient, where following the gradient does not successfully
*

Equal contribution 1 Massachusetts Institute of Technology 2 University of California, Berkeley.
Correspondence
to: Anish Athalye <aathalye@mit.edu>, Nicholas Carlini
<npc@berkeley.edu>.

optimize the loss, iterative optimization-based methods cannot succeed. We identify three types of obfuscated gradients:
• Shattered gradients are nonexistent or incorrect
gradients caused either intentionally through nondifferentiable operations or unintentionally through
numerical instability.
• Stochastic gradients are gradients that depend on testtime entropy unavailable to the attacker.
• Vanishing/exploding gradients in very deep or recurrent computation with long-term dependencies result
in an unusable gradient.
We propose new techniques to overcome obfuscated gradients caused by these three phenomenon. We address gradient shattering with a new attack technique we call Backward
Pass Differentiable Approximation, where we approximate
derivatives by computing the forward pass normally and
compute the backward pass through a differentiable approximation of the function. We compute gradients of randomized defenses by applying Expectation Over Transformation (Athalye et al., 2017). We solve vanishing/exploding
gradients through reparameterization and optimize over a
space where gradients do not explode/vanish.
To investigate the prevalence of obfuscated gradients and understand the applicability of these attack techniques, we use
the ICLR 2018 non-certified defenses that argue white-box
robustness as a case study. We find that obfuscated gradients
are a common occurrence, with 7 of 8 defenses relying on
this phenomenon. Applying the new attack techniques we
develop, we overcome obfuscated gradients and circumvent
6 of them completely, and 1 partially. Along with this, we
offer an analysis of the evaluations performed in the papers.
Additionally, we hope to provide researchers with a common
baseline of knowledge, description of attack techniques,
and common evaluation pitfalls, so that future defenses can
avoid falling vulnerable to these same attacks.
To promote reproducible research, we release our reimplementation of each of these defenses, along with implementations of our attacks for each. 1
1

https://github.com/anishathalye/obfuscated-gradients

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples

2. Preliminaries
2.1. Notation
We consider a neural network f (·) used for classification
where f (x)i represents the probability image x corresponds
to label i. We classify images, represented as x ∈ [0, 1]w·h·3
for a 3-color image of width w and height h. We use f j (·)
to refer to layer j of the neural network, and f 1..j (·) the
composition of layers 1 through j. We denote the classification of the network as c(x) = arg maxi f (x)i where the
true label of image x is written c∗ (x).
2.2. Adversarial Examples
Given an image x and classifier f (·), an adversarial example
(Szegedy et al., 2013) x0 satisfies two properties: D(x, x0 )
is small for some distance metric D, and c(x0 ) 6= c∗ (x).
That is, for images, x and x0 appear visually similar but x0
is classified incorrectly.
For this paper we use the `∞ and `2 distortion metrics to
measure similarity. Two images which have a small distortion under either of these metrics will appear visually
identical. We report `∞ distance in the normalized [0, 1]
space, so that a distortion of 0.031 corresponds to 8/256,
and `2 distance as the total root-mean-square distortion normalized by the total number of pixels.
2.3. Datasets & Models
We evaluate these defenses on the same datasets on which
they claim robustness.If a defense argues security on MNIST
and any other dataset, we only circumvent the defense on
the larger dataset. On MNIST and CIFAR-10, we evaluate defenses over the entire test set and generate untargeted adversarial examples. On ImageNet, we evaluate over
1000 randomly selected images in the test set, and construct
targeted adversarial examples with randomly selected target classes. Generating targeted adversarial examples is
a strictly harder problem which we believe is also a more
meaningful metric, especially for this dataset.
We use standard models for each dataset. For MNIST we
use a standard 5-layer convolutional neural network which
reaches 99.3% accuracy. On CIFAR-10 we train a wide
resnet (Zagoruyko & Komodakis, 2016; He et al., 2016)
to 95% accuracy. For ImageNet we use the InceptionV3
(Szegedy et al., 2016) network which reaches 78.0% top-1
and 93.9% top-5 accuracy.
2.4. Threat Models
Prior work considers adversarial examples in a number of
threat models that can be broadly classified into two categories: white-box and black-box. In this paper, we consider defenses designed for the white-box setting, where the

adversary has full access to the neural network classifier
(architecture and weights) and defense, but not test-time
randomness (only the distribution).
2.5. Attack Methods
We construct adversarial examples with iterative
optimization-based methods. At a high level, for a
given instance x, these attacks attempt to search for a δ
such that c(x + δ) 6= c∗ (x) either minimizing kδk, or
maximizing the classification loss on f (x + δ). To generate
`∞ bounded adversarial examples we use Projected
Gradient Descent (PGD) confined to a specified `∞ ball;
for `2 , we use the Lagrangian relaxation of Carlini &
Wagner (2017c). The specific choice of optimizer is far less
important than choosing to use iterative optimization-based
methods (Madry et al., 2018).

3. Obfuscated Gradients
A defense is said to cause gradient masking if it “does
not have useful gradients” for generating adversarial examples (Papernot et al., 2017), and is known to be an incomplete defense to adversarial examples (Papernot et al., 2017;
Tramèr et al., 2018). Despite this, we observe that 7 of the
ICLR defenses rely on this effect explicitly.
To contrast from existing defenses which cause gradient
masking by learning to break gradient descent (e.g., by
learning to make the gradients point the wrong direction
(Tramèr et al., 2018)), we refer to the case where defenses
are designed in such a way that the constructed defense
necessarily causes gradient masking as obfuscated gradients. We discover three ways in which defenses obfuscate
gradients; we briefly define and discuss each of them.
Shattered Gradients are caused when a defense is nondifferentiable, introduces numeric instability, or otherwise
causes a gradient to be nonexistent or incorrect. Defenses
that cause gradient shattering often do so unintentionally,
by introducing operations that are differentiable but where
following the gradient does not maximize classification loss.
Stochastic Gradients are caused by randomized defenses,
where either the network itself is randomized or the input
is randomized before being fed to the classifier, and so the
gradients become randomized. This causes methods using a
single sample of the randomness to incorrectly estimate the
true gradient.
Exploding & Vanishing Gradients are often caused by defenses that consist of multiple iterations of neural network
evaluation, feeding the output of one computation as the
input of the next. This type of computation, when unrolled,
can be viewed as an extremely deep neural network evaluation, which can cause vanishing/exploding gradients.

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples

3.1. Identifying Obfuscated & Masked Gradients

4.1. Backward Pass Differentiable Approximation

As we observe, some defenses intentionally (by design)
break gradient descent and cause obfuscated gradients. However, other defenses also (by design) break gradient descent,
even if it was not the intention of the designer. We discuss below characteristic behaviors of defenses which have
caused this to occur. While these behaviors may not perfectly characterize all cases of masked gradients, we find
that every defense we are aware of that masks gradients
exhibits at least one of these behaviors.

Shattered gradients, caused either unintentionally, e.g. by
numerical instability, or intentionally, e.g. by using nondifferentiable operations, result in nonexistent or incorrect
gradients. To attack defenses where gradients are not readily
available, we introduce a technique we call Backward Pass
Differentiable Approximation (BPDA) 2 .

One-step attacks perform better than iterative attacks.
Iterative optimization-based attacks applied in a white-box
setting are strictly stronger than single-step attacks and
should give strictly superior performance. If single-step
methods give performance superior to iterative methods, it
is likely that the iterative attack is becoming stuck in its
optimization search at a local minimum.

Black-box attacks are better than white-box attacks.
The black-box threat model is a strict subset of the whitebox threat model, and so attacks in the white-box setting
should perform better; however, if a defense is obfuscating
gradients, then often times black-box attacks which do not
use the gradient will perform better than white-box attacks
(Papernot et al., 2017).

Unbounded attacks do not reach 100% success. With
unbounded distortion, any classifier should have 0% robustness to adversarial examples (as long as the classifier is not
a constant function). If attacks do not reach 100% success,
this indicates the defense is defeating the attack in a subtle
manner and may not actually be increasing robustness.

Random sampling finds adversarial examples. If bruteforce random search (e.g., randomly sampling 105 or more
points) within some -ball successfully finds adversarial
examples when gradient methods do not, the defense is
likely to be masking gradients.

4. Attack Techniques
Generating adversarial examples through optimizationbased methods requires useful gradients obtained through
backpropagation (Rumelhart et al., 1986). Many defenses
therefore either intentionally or unintentionally cause gradient descent to fail because of obfuscated gradients caused
by gradient shattering, stochastic gradients, or vanishing/exploding gradients. We discuss a number of techniques
that we develop to overcome obfuscated gradients.

4.1.1. S PECIAL C ASE : S TRAIGHT-T HROUGH
E STIMATOR
As a special case, we first discuss what amounts to the
straight-through estimator (Bengio et al., 2013) applied to
constructing adversarial examples.
Many non-differentiable defenses can be expressed as follows: given a pre-trained classifier f (·), construct a preprocessor g(·) and let the secured classifier fˆ(x) = f (g(x))
where the preprocessor g(·) satisfies g(x) ≈ x (e.g., such a
g(·) may perform image denoising to remove the adversarial perturbation, as in Guo et al. (2018)). If g(·) is smooth
and differentiable, then computing gradients through the
combined network fˆ is often sufficient to circumvent the
defense (Carlini & Wagner, 2017b). However, recent work
has constructed functions g(·) which are neither smooth
nor differentiable, and therefore can not be backpropagated
through to generate adversarial examples with a white-box
attack.
We introduce a new attack that we call Backward Pass Differentiable Approximation. Because g is constructed with the
property that g(x) ≈ x, we can approximate its derivative as
the derivative of the identity function: ∇x g(x) ≈ ∇x x = 1.
Therefore, we can approximate the derivative of f (g(x)) at
the point x̂ as:
∇x f (g(x))|x=x̂ ≈ ∇x f (x)|x=g(x̂)
This allows us to compute gradients and therefore mount a
white-box attack. Conceptually, this attack is simple. We
perform forward propagation through the neural network as
usual, but on the backward pass, we replace g(·) with the
identity function. In practice, the implementation can be expressed in an even simpler way: we approximate ∇x f (g(x))
by evaluating ∇x f (x) at the point g(x). This gives us an
approximation of the true gradient, and while not perfect, is
sufficiently useful that when averaged over many iterations
of gradient descent still generates an adversarial example.
4.1.2. G ENERALIZED ATTACK : BPDA
While the above attack is effective for a simple class of
networks expressible as f (g(x)) when g(x) ≈ x, it is not
fully general. We now generalize the above approach.
2
The BPDA approach can be used on an arbitrary network,
even if it is already differentiable, to obtain a more useful gradient.

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples

Let f (·) = f 1...j (·) be a neural network, and let f i (·)
be a non-differentiable layer. To approximate ∇x f (x),
we first find a differentiable approximation g(x) such that
g(x) ≈ f i (x). Then, we can approximate ∇x f (x) by performing the forward pass through f (·) (and in particular,
computing a forward pass through f i (x)), but on the backward pass, replacing f i (x) with g(x). Note that we perform
this replacement only on the backward pass.
As long as the two functions are similar, we find that the
slightly inaccurate gradients still prove useful in constructing an adversarial example.
We have found applying BPDA is often necessary: replacing f i (·) with g(·) on both passes either is completely ineffective (e.g., with Song et al. (2018)) or many times less
effective (e.g. with Buckman et al. (2018)).
4.2. Attacking Randomized Classifiers
Stochastic gradients arise when using randomized transformations to the input before feeding it to the classifier or
when using a stochastic classifier. When using optimizationbased attacks on defenses that employ these techniques, it is
necessary to estimate the gradient of the stochastic function.
Expectation over Transformation. For defenses that employ randomized transformations to the input, we apply Expectation over Transformation (EOT) (Athalye et al., 2017)
to correctly compute the gradient over the expected transformation to the input.
When attacking a classifier f (·) that first randomly transforms its input according to a function t(·) sampled from a
distribution of transformations T , EOT optimizes the expectation over the transformation Et∼T f (t(x)). The optimization problem can be solved by gradient descent, noting that
∇Et∼T f (t(x)) = Et∼T ∇f (t(x)), differentiating through
the classifier and transformation, and approximating the
expectation with samples at each gradient descent step.
4.3. Reparameterization
We solve vanishing/exploding gradients by reparameterization. Assume we are given a classifier f (g(x)) where g(·)
performs some optimization loop to transform the input x
to a new input x̂. Often times, this optimization loop means
that differentiating through g(·), while possible, yields exploding or vanishing gradients.
To resolve this, we make a change-of-variable x = h(z)
for some function h(·) such that g(h(z)) = h(z) for all
z, but h(·) is differentiable. For example, if g(·) projects
samples to some manifold in a specific manner, we might
construct h(z) to return points exclusively on the manifold.
This allows us to compute gradients through f (h(z)) and
thereby circumvent the defense.

Defense

Dataset

Distance

Accuracy

Buckman et al. (2018)
Ma et al. (2018)
Guo et al. (2018)
Dhillon et al. (2018)
Xie et al. (2018)
Song et al. (2018)
Samangouei et al.
(2018)

CIFAR
CIFAR
ImageNet
CIFAR
ImageNet
CIFAR
MNIST

0.031 (`∞ )
0.031 (`∞ )
0.005 (`2 )
0.031 (`∞ )
0.031 (`∞ )
0.031 (`∞ )
0.005 (`2 )

0%∗
5%
0%∗
0%
0%∗
9%∗
55%

Madry et al. (2018)

CIFAR

0.031 (`∞ )

47%

Table 1. Summary of Results: Seven of eight defense techniques
accepted to ICLR 2018 cause obfuscated gradients and are vulnerable to our attacks. (Defenses denoted with ∗ also propose
combining adversarial training; we report here the defense alone,
see §5 for full numbers.)

5. Case Study: ICLR 2018 Defenses
As a case study for evaluating the prevalence of obfuscated
gradients, we study the ICLR 2018 non-certified defenses
that argue robustness in a white-box threat model. We find
that seven of these eight defenses relies on this phenomenon
to argue security, and we demonstrate that our techniques
can completely circumvent six of those (and partially circumvent one) that rely on obfuscated gradients. We omit
two defenses with provable security claims (Aditi Raghunathan, 2018; Aman Sinha, 2018) and one that only argues
black-box security (Tramèr et al., 2018). We include one
paper, Ma et al. (2018), that was not proposed as a defense
per se, but suggests a method to detect adversarial examples.
There is an asymmetry in attacking defenses versus constructing robust defenses: to show a defense can be bypassed, it is only necessary to demonstrate one way to do
so; in contrast, a defender must show no attack succeeds.
Table 1 summarizes our results. Of the 8 accepted papers, 7
rely on obfuscated gradients. Two of these defenses argue
robustness on ImageNet, a much harder task than CIFAR10; and one argues robustness on MNIST, a much easier
task than CIFAR-10. As such, comparing defenses across
datasets is difficult.
5.1. Non-obfuscated Gradients
5.1.1. A DVERSARIAL T RAINING
Defense Details. Originally proposed by Szegedy et al.
(2013), adversarial training is a conceptually simple process:
train on adversarial examples until the model learns to classify them correctly. Given training data X and loss function
`(·), standard training chooses network weights θ as
θ∗ = arg min Ex∈X `(x; Fθ ).
θ

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples

Adversarial training instead chooses an -ball and solves the
min-max formulation


θ∗ = arg min Ex∈X
max `(x + δ; Fθ ) .
θ

δ∈[−,]N

To approximately solve this formulation, Madry et al. (2018)
solve the inner maximization problem by generating adversarial examples the training data using projective gradient
descent.
Discussion. The evaluation the authors perform for this
defense tests for all of the characteristic behaviors of obfuscated gradients that we list. Additionally, we believe
this approach does not cause obfuscated gradients: our experiments with iterative optimization-based attacks do succeed with some probability (but do not invalidate the claims
made in the paper). However, we note that (1) adversarial retraining has been shown to be difficult at ImageNet
scale (Kurakin et al., 2016b), and (2) training exclusively on
`∞ adversarial examples provides only limited robustness
to adversarial examples under other distortion metrics.
5.2. Gradient Shattering
5.2.1. T HERMOMETER E NCODING
Defense Details. In contrast to prior work (Szegedy et al.,
2013) which viewed adversarial examples as “blind spots”
in neural networks, Goodfellow et al. (2014b) argue that the
reason adversarial examples exist is that neural networks behave in a largely linear manner. The purpose of thermometer
encoding is to break this linearity.
Given an image x, for each pixel color xi,j,c , the l-level
thermometer encoding τ (xi,j,c ) is a l-dimensional vector
where
(
1 if xi,j,c > k/l
τ (xi,j,c )k =
.
0 otherwise
For example, for a 10-level thermometer encoding, we have
τ (0.66) = 1111110000. Training networks using thermometer encoding is identical to normal training.
Due to the discrete nature of thermometer encoded values, it is not possible to directly perform gradient descent
on a thermometer encoded neural network. The authors
therefore construct Logit-Space Projected Gradient Ascent
(LS-PGA) as an attack over the discrete thermometer encoded inputs. Using this attack, the authors perform the
adversarial training of Madry et al. (2018) on thermometer
encoded networks.
On CIFAR-10, just performing thermometer encoding was
found to give 50% accuracy within  = 0.031 under `∞
distortion. By performing adversarial training with 7 steps
of LS-PGA, robustness increased to 80%.

Discussion. While the intention behind this defense is to
break the local linearity of neural networks, we find that
this defense in fact causes gradient shattering. This can
be observed through their black-box attack evaluation: adversarial examples generated on a standard adversarially
trained model transfer to a thermometer encoded model reducing the accuracy to 67%, well below the 80% robustness
to the white-box iterative attack.
Evaluation. We use the BPDA approach from Section 4.1.2, where we let g(x) = τ (x). Observe that if
we define
τ̂ (xi,j,c )k = min(max(xi,j,c − k/l, 0), 1)
then

τ (xi,j,c )k = floor(τ̂ (xi,j,c )k )

so we can let h(x) = τ̂ (x) and replace the backwards pass
with the function h(·).
LS-PGA reduces model accuracy to 50% on a thermometerencoded model trained without adversarial training
(bounded by  = 0.031). In contrast, we achieve 1% model
accuracy with the lower  = 0.015 (and 0% with  = 0.031).
This shows no measurable improvement from standard models, trained without thermometer encoding.
When we attack a thermometer-encoded adversarially
trained model 3 , we are able to reproduce the 80% accuracy at  = 0.031 claim against LS-PGA. However, our
attack reduces model accuracy to 20%. This is significantly
weaker than the original Madry et al. (2018) model that
does not use thermometer encoding. Because this model is
trained against the (comparatively weak) LS-PGA attack, it
is unable to adapt to the stronger attack we present above.
5.2.2. I NPUT T RANSFORMATIONS
Defense Details. Guo et al. (2018) propose five input
transformations to counter adversarial examples.
As a baseline, the authors evaluate image cropping and
rescaling, bit-depth reduction, and JPEG compression as
baseline transformations. Then the authors suggest two new
transformations:
• Randomly drop pixels, and restore them by performing
total variance minimization.
• Image quilting: Reconstruct images by replacing all
5 × 5 patches with patches from “clean” images, using
minimum graph cuts in overlapping boundary regions
to remove edge artifacts.
The authors explore different combinations of input transformations along with different underlying ImageNet classifiers, including adversarially trained models. They find
3
That is, a thermometer encoded model that is trained using
the approach of (Madry et al., 2018).

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples

1.0

Baseline
Thermometer
Adv. Train
Adv. Therm

Model Accuracy

0.8
0.6
0.4

Analysis Overview. Instead of actively attempting to attack the detection method, we find that LID is not able to
detect high confidence adversarial examples (Carlini & Wagner, 2017a) generated oblivious to the details defense. A
full discussion of this defense is given in Appendix A.

0.2
0.0

0.00

0.01
0.02
Perturbation Magnitude

0.03

Figure 1. Model accuracy versus distortion (under `∞ ). Adversarial training increases robustness to 50% at  = 0.031; thermometer
encoding by itself provides limited value, and when coupled with
adversarial training performs worse than adversarial training alone.

that input transformations provide protection even with a
vanilla classifier, providing varying degrees of robustness
for varying transformations and `2 perturbation budgets.
Discussion. The authors find that a ResNet-50 classifier
provides a varying degree of accuracy for each of the five
proposed input transformationsunder the strongest attack
with a normalized `2 dissimilarity of 0.01, with the strongest
defenses achieving over 60% top-1 accuracy. We observe
similar results when evaluating an InceptionV3 classifier.
The authors do not succeed in white-box attacks, crediting lack of access to test-time randomness as “particularly
crucial in developing strong defenses” (Guo et al., 2018). 4
Evaluation. It is possible to bypass each defense independently. We circumvent image cropping and rescaling
with a direct application of Expectation Over Transformation (Athalye et al., 2017). To circumvent bit-depth reduction, JPEG compression, total variance minimization, and
image quilting, we use BPDA to approximate the backward
pass with the identity function. With our attack, model
accuracy drops to 0% for the strongest defense under the
smallest perturbation budget considered in Guo et al. (2018),
a root-mean-square perturbation of 0.05 (corresponding to a
“normalized” `2 perturbation as defined in Guo et al. (2018)
of 0.01).
5.2.3. L OCAL I NTRINSIC D IMENSIONALITY (LID)
LID is a general-purpose metric that measures the distance
from an input to its neighbors. Ma et al. (2018) propose
4

using LID to characterize properties of adversarial examples.
The authors emphasize that this classifier is not intended as a
defense against adversarial examples 5 , however the authors
argue that it is a robust method for detecting adversarial
examples that is not easy to evade.

This defense may be stronger in a threat model where the
adversary does not have complete information about the exact
quilting process used (Personal communication with authors).

5.3. Stochastic Gradients
5.3.1. S TOCHASTIC ACTIVATION P RUNING (SAP)
Defense Details. SAP (Dhillon et al., 2018) introduces
randomness into the evaluation of a neural network to defend against adversarial examples. SAP randomly drops
some neurons of each layer f i to 0 with probability proportional to their absolute value. That is, SAP essentially
applies dropout at each layer where instead of dropping with
uniform probability, nodes are dropped with a weighted distribution. Values which are retained are scaled up (as is
done in dropout) to retain accuracy. Applying SAP decreases clean classification accuracy slightly, with a higher
drop probability decreasing accuracy, but increasing robustness. We study various levels of drop probability and find
they lead to similar robustness numbers.
Discussion. The authors only evaluate SAP by taking a
single step in the gradient direction (Dhillon et al., 2018).
While taking a single step in the direction of the gradient
is effective on non-randomized neural networks, when randomization is used, computing the gradient with respect to
one sample of the randomness is ineffective.
Evaluation. To resolve this difficult, we estimate the gradients by computing the expectation over instantiations of
randomness. At each iteration of gradient descent, instead
of taking a step in the direction of ∇x f (x) we move in the
Pk
direction of i=1 ∇x f (x) where each invocation is randomized with SAP. We have found that choosing k = 10
provides useful gradients. We additionally had to resolve
a numerical instability when computing gradients: this defense caused computing a backward pass to cause exploding
gradients due to division by numbers very close to 0. We resolve this by clipping gradients or through stable numerical
techniques.
With these approaches, we are able to reduce SAP model
accuracy to 9% at  = .015, and 0% at  = 0.031. If
we consider an attack successful only when an example
5

Personal communication with authors.

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples

is classified incorrectly 10 times out of 10 (and consider it
correctly classified if it is ever classified as the correct label),
model accuracy is below 10% with  = 0.031.
5.3.2. M ITIGATING THROUGH R ANDOMIZATION
Defense Details. (Xie et al., 2018) propose to defend
against adversarial examples by adding a randomization
layer before the input to the classifier. For a classifier that
takes a 299 × 299 input, the defense first randomly rescales
the image to a r × r image, with r ∈ [299, 331), and then
randomly zero-pads the image so that the result is 331×331.
The output is then fed to the classifier.
Discussion. The authors consider three attack scenarios:
vanilla attack (an attack on the original classifier), singlepattern attack (an attack assuming some fixed randomization
pattern), and ensemble-pattern attack (an attack over a small
ensemble of fixed randomization patterns). The authors
strongest attack reduces InceptionV3 model accuracy to
32.8% top-1 accuracy (over images that were originally
classified correctly).
The authors dismiss a stronger attack over larger choices
of randomness, stating that it would be “computationally
impossible” (emphasis ours) and that such an attack “may
not even converge” (Xie et al., 2018).
Evaluation. We find the authors ensemble attack overfits
to the ensemble with fixed randomization. We bypass this
defense by applying EOT (Athalye et al., 2017), optimizing
over the (in this case, discrete) distribution of transformations T and minimizing Et∼T f (t(x)).

an -ball of the input image.
Discussion. The authors evaluate PixelDefend on CIFAR10 over various classifiers, perturbation budgets, and attacks.
With a maximum `∞ perturbation of  = 0.031 on CIFAR10, PixelDefend claims 46% accuracy (with a vanilla ResNet
classifier). The authors dismiss the possibility of end-to-end
attacks on PixelDefend due to the difficulty of differentiating
through an unrolled version of PixelDefend due to vanishing
gradients and computation cost.
Evaluation. We sidestep the problem of computing gradients through an unrolled version of PixelDefend by approximating gradients with BPDA, approximating the backward
pass with the identity function, and we successfully mount
an end-to-end attack using this technique 6 . With this attack,
we can reduce the accuracy of a naturally trained classifier
which achieves 95% accuracy to 9% with a maximum `∞
perturbation of  = 0.031. Combining Madry et al. (2018)
with PixelDefend provides no additional robustness over
just using the adversarially trained classifier.
5.4.2. D EFENSE -GAN
Defense-GAN (Samangouei et al., 2018) uses a Generative
Adversarial Network (Goodfellow et al., 2014a) to project
samples onto the manifold of the generator before classifying them. That is, the intuition behind this defense is
nearly identical to PixelDefend using a GAN instead of a
PixelCNN. We therefore summarize results here and present
the full details in Appendix B.

5.4. Vanishing & Exploding Gradients

Analysis Overview. DefenseGAN is not effective on CIFAR; so we must evaluate on MNIST. We find that adversarial examples exist on the manifold as described by the
generator. However, while this attack would defeat a perfect
projector mapping x to it’s nearest point G(z), the actual
imperfect projection approach taken by Defense-GAN will
not identify these points. We therefore construct a second
attack using BPDA to evade Defense-GAN, although at only
a 45% success rate under reasonable `2 perturbation bounds.

5.4.1. P IXEL D EFEND

6. Discussion

We approximate the gradient of the above by sampling and
differentiating through the transformation. Using this attack,
even if we consider the attack successful only when an
example is classified incorrectly 10 times out of 10, we can
reduce the accuracy of the classifier from 32.8% to 0.0%
with a maximum `∞ perturbation of  = 0.031.

Defense Details. Song et al. (2018) propose using a PixelCNN generative model to project a potential adversarial
example back onto the data manifold before feeding it into
a classifier. The authors argue that adversarial examples
mainly lie in the low-probability region of the training distribution. PixelDefend “purifies” adversarially perturbed
images by projecting them back onto the data manifold
through the use of a PixelCNN generative model, and then
it feeds the resulting image through an unmodified classifier. PixelDefend uses a greedy decoding procedure to
approximate finding the highest probability example within

Having demonstrated attacks on these seven defenses, we
now take a step back and discuss the method of evaluating a
defense against adversarial examples.
The papers we study use a variety of approaches in evaluating robustness of the proposed defenses. Synthesizing
these, we list what we believe to be the most important
6
In place of a PixelCNN, due to the availability of a pre-trained
model, we use a PixelCNN++ (Salimans et al., 2017) and discretize
the mixture of logistics to produce a 256-way softmax over possible
pixel values.

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples

points to keep in mind while building and evaluating defenses. Much of what we describe below has been given in
prior work (Carlini & Wagner, 2017a; Madry et al., 2018);
we repeat these points here and offer our own perspective
for completeness.
6.1. Defining a (realistic) threat model
A threat model specifies the conditions under which the defense is aiming to be secure: a precise threat model allows
for a precise understanding of the setting under which the
defense is meant to work. Prior work has used words including white-box, grey-box, black-box, no-box to describe
slightly different threat models, often overloading the same
word.
Instead of attempting to, yet again, redefine the vocabulary,
we enumerate the various aspects of a defense that might be
revealed to the adversary or held secret to the defender:
• Model architecture and Model weights.
• Training algorithm and Training data.
• For defenses that involve randomness, whether the
adversary knows the exact sequence of random values
that will be chosen, or only the distribution.
• If assuming the adversary does not know the model
architecture and weights, if query access is allowed. If
so, if the model output is the logits, probability vector,
or predicted label (i.e., arg max).
While there are some aspects of a defense that might be
held secret, threat models should not contain unrealistic
constraints. We believe any compelling threat model should
at the very least grant knowledge of the model architecture,
training algorithm, and allow query access.
We do not believe it is meaningful to restrict the computational power of an adversary. If two defenses are equally
robust but generating adversarial examples on one takes one
second and another takes ten seconds, the robustness has
not increased.
6.2. Making specific, testable claims
Specific, testable claims in a clear threat model precisely
convey the claimed robustness of a defense. For example,
a defense might claim 90% accuracy on `∞ adversarial
examples of distortion at most  = 0.031, or might claim
that mean `2 distortion to adversarial examples increases by
a factor of two from a baseline model to a secured model
(in which case, the baseline should also be clearly defined).
A defense can never achieve complete robustness against
unbounded attacks: with unlimited distortion any image can
be converted into any other, yielding 100% “success”.
A defense being specified completely, with all hyperpa-

rameters given, is a prerequisite for claims to be testable.
Releasing source code and a pre-trained model along with
the paper describing a specific threat model and robustness
claims is perhaps the most useful method of making testable
claims. Four of the defenses we study made complete source
code available (Madry et al., 2018; Ma et al., 2018; Guo
et al., 2018; Xie et al., 2018).
6.3. Evaluating against adaptive attacks
A secure defense is robust not only against existing attacks
but also against all possible attacks within the specified
threat model. While certified defenses succeed in reasoning
about all possible attacks, it is often challenging to do so.
However, actively evaluating a defense with new defenseaware attacks crafted specifically to circumvent the defense
helps justify claims of security.
An adaptive attack is one that is constructed after a defense
has been completely specified, where the adversary takes
advantage of knowledge of the defense and is only restricted
by the threat model. If a defense can be circumvented by
an adaptive attack, giving a way to prevent that specific
attack (e.g. by tweaking a hyperparameter) does not imply
robustness. If a defense is modified after an evaluation, an
adaptive attack is one that considers knowledge of the new
defense. In this way, concluding an evaluation with a final
adaptive attack can be seen as analogous to evaluating a
model on the test data.

7. Conclusion
Constructing defenses to adversarial examples requires defending against not only existing attacks but also future
attacks that may be developed. In this paper, we identify
obfuscated gradients, a phenomenon exhibited by certain
defenses that makes standard gradient-based methods fail
to generate adversarial examples. We develop three attack
techniques to bypass three different types of obfuscated gradients. To evaluate the applicability of our techniques, we
use the ICLR 2018 defenses as a case study, circumventing
seven of eight accepted defenses.
More generally, we hope that future work will be able to
avoid relying on obfuscated gradients for perceived robustness and use our evaluation approach to detect when this
occurs. Defending against adversarial examples is an important area of research and we believe performing a thorough
evaluation is a critical step that can not be overlooked.

Acknowledgements
We are grateful to Aleksander Madry, Andrew Ilyas, and
Aditi Raghunathan for helpful comments on an early draft
of this paper. We thank Bo Li, Xingjun Ma, Laurens van der

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples

Maaten, Aurko Roy, Yang Song, and Cihang Xie for useful
discussion and insights on their defenses.

adversarial examples. arXiv preprint arXiv:1711.08478,
2017b.

This work was partially supported by the National Science
Foundation through award CNS-1514457, Qualcomm, and
the Hewlett Foundation through the Center for Long-Term
Cybersecurity.

Carlini, Nicholas and Wagner, David. Towards evaluating
the robustness of neural networks. In IEEE Symposium
on Security & Privacy, 2017c.

References
Aditi Raghunathan, Jacob Steinhardt, Percy Liang. Certified defenses against adversarial examples.
International Conference on Learning Representations,
2018. URL https://openreview.net/forum?
id=Bys4ob-Rb.
Aman Sinha, Hongseok Namkoong, John Duchi. Certifiable distributional robustness with principled adversarial
training. International Conference on Learning Representations, 2018. URL https://openreview.net/
forum?id=Hk6kPgZA-.
Amsaleg, Laurent, Chelly, Oussama, Furon, Teddy, Girard,
Stéphane, Houle, Michael E, Kawarabayashi, Ken-ichi,
and Nett, Michael. Estimating local intrinsic dimensionality. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data
Mining, pp. 29–38. ACM, 2015.
Athalye, Anish, Engstrom, Logan, Ilyas, Andrew, and
Kwok, Kevin. Synthesizing robust adversarial examples.
arXiv preprint arXiv:1707.07397, 2017.
Bengio, Yoshua, Léonard, Nicholas, and Courville, Aaron.
Estimating or propagating gradients through stochastic
neurons for conditional computation. arXiv preprint
arXiv:1308.3432, 2013.
Biggio, Battista, Corona, Igino, Maiorca, Davide, Nelson,
Blaine, Šrndić, Nedim, Laskov, Pavel, Giacinto, Giorgio,
and Roli, Fabio. Evasion attacks against machine learning
at test time. In Joint European Conference on Machine
Learning and Knowledge Discovery in Databases, pp.
387–402. Springer, 2013.
Buckman, Jacob, Roy, Aurko, Raffel, Colin, and Goodfellow, Ian. Thermometer encoding: One hot way to
resist adversarial examples. International Conference
on Learning Representations, 2018. URL https://
openreview.net/forum?id=S18Su--CW. accepted as poster.

Dhillon, Guneet S., Azizzadenesheli, Kamyar, Bernstein,
Jeremy D., Kossaifi, Jean, Khanna, Aran, Lipton,
Zachary C., and Anandkumar, Animashree. Stochastic
activation pruning for robust adversarial defense. International Conference on Learning Representations,
2018. URL https://openreview.net/forum?
id=H1uR4GZRZ. accepted as poster.
Goodfellow, Ian, Pouget-Abadie, Jean, Mirza, Mehdi, Xu,
Bing, Warde-Farley, David, Ozair, Sherjil, Courville,
Aaron, and Bengio, Yoshua. Generative adversarial nets.
In Advances in neural information processing systems,
pp. 2672–2680, 2014a.
Goodfellow, Ian J, Shlens, Jonathon, and Szegedy, Christian.
Explaining and harnessing adversarial examples. arXiv
preprint arXiv:1412.6572, 2014b.
Gulrajani, Ishaan, Ahmed, Faruk, Arjovsky, Martin, Dumoulin, Vincent, and Courville, Aaron. Improved training
of wasserstein gans. arXiv preprint arXiv:1704.00028,
2017.
Guo, Chuan, Rana, Mayank, Cisse, Moustapha, and van der
Maaten, Laurens. Countering adversarial images using input transformations. International Conference
on Learning Representations, 2018. URL https:
//openreview.net/forum?id=SyJ7ClWCb. accepted as poster.
He, Kaiming, Zhang, Xiangyu, Ren, Shaoqing, and Sun,
Jian. Deep residual learning for image recognition. In
Proceedings of the IEEE conference on computer vision
and pattern recognition, pp. 770–778, 2016.
Ilyas, Andrew, Jalal, Ajil, Asteri, Eirini, Daskalakis, Constantinos, and Dimakis, Alexandros G. The robust manifold defense: Adversarial training using generative models. arXiv preprint arXiv:1712.09196, 2017.
Kurakin, Alexey, Goodfellow, Ian, and Bengio, Samy. Adversarial examples in the physical world. arXiv preprint
arXiv:1607.02533, 2016a.

Carlini, Nicholas and Wagner, David. Adversarial examples
are not easily detected: Bypassing ten detection methods.
AISec, 2017a.

Kurakin, Alexey, Goodfellow, Ian J., and Bengio, Samy.
Adversarial machine learning at scale. arXiv preprint
arXiv:1611.01236, 2016b.

Carlini, Nicholas and Wagner, David. Magnet and “efficient
defenses against adversarial attacks” are not robust to

Ma, Xingjun, Li, Bo, Wang, Yisen, Erfani, Sarah M.,
Wijewickrema, Sudanthi, Schoenebeck, Grant, Houle,

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples

Michael E., Song, Dawn, and Bailey, James. Characterizing adversarial subspaces using local intrinsic dimensionality. International Conference on Learning Representations, 2018. URL https://openreview.net/
forum?id=B1gJ1L2aW. accepted as oral presentation.

Tramèr, Florian, Kurakin, Alexey, Papernot, Nicolas, Goodfellow, Ian, Boneh, Dan, and McDaniel, Patrick. Ensemble adversarial training: Attacks and defenses. International Conference on Learning Representations,
2018. URL https://openreview.net/forum?
id=rkZvSe-RZ. accepted as poster.

Madry, Aleksander, Makelov, Aleksandar, Schmidt, Ludwig, Tsipras, Dimitris, and Vladu, Adrian. Towards
deep learning models resistant to adversarial attacks.
International Conference on Learning Representations,
2018. URL https://openreview.net/forum?
id=rJzIBfZAb. accepted as poster.

Xie, Cihang, Wang, Jianyu, Zhang, Zhishuai, Ren, Zhou,
and Yuille, Alan. Mitigating adversarial effects through
randomization. International Conference on Learning
Representations, 2018. URL https://openreview.
net/forum?id=Sk9yuql0Z. accepted as poster.

Papernot, Nicolas, McDaniel, Patrick, Goodfellow, Ian,
Jha, Somesh, Celik, Z. Berkay, and Swami, Ananthram. Practical black-box attacks against machine
learning. In Proceedings of the 2017 ACM on Asia
Conference on Computer and Communications Security, ASIA CCS ’17, pp. 506–519, New York, NY,
USA, 2017. ACM. ISBN 978-1-4503-4944-4. doi:
10.1145/3052973.3053009. URL http://doi.acm.
org/10.1145/3052973.3053009.
Rumelhart, David E., Hinton, Geoffrey E., and Williams,
Ronald J. Learning representations by back-propagating
errors. Nature, 323:533–536, 1986.
Salimans, Tim, Karpathy, Andrej, Chen, Xi, and Kingma,
Diederik P. Pixelcnn++: A pixelcnn implementation with
discretized logistic mixture likelihood and other modifications. In ICLR, 2017.
Samangouei, Pouya, Kabkab, Maya, and Chellappa, Rama.
Defense-gan: Protecting classifiers against adversarial attacks using generative models. International Conference
on Learning Representations, 2018. URL https://
openreview.net/forum?id=BkJ3ibb0-. accepted as poster.
Song, Yang, Kim, Taesup, Nowozin, Sebastian, Ermon,
Stefano, and Kushman, Nate. Pixeldefend: Leveraging
generative models to understand and defend against adversarial examples. International Conference on Learning
Representations, 2018. URL https://openreview.
net/forum?id=rJUYGxbCW. accepted as poster.
Szegedy, Christian, Zaremba, Wojciech, Sutskever, Ilya,
Bruna, Joan, Erhan, Dumitru, Goodfellow, Ian, and Fergus, Rob. Intriguing properties of neural networks. ICLR,
2013.
Szegedy, Christian, Vanhoucke, Vincent, Ioffe, Sergey,
Shlens, Jon, and Wojna, Zbigniew. Rethinking the inception architecture for computer vision. In Proceedings
of the IEEE Conference on Computer Vision and Pattern
Recognition, pp. 2818–2826, 2016.

Zagoruyko, Sergey and Komodakis, Nikos. Wide residual
networks. arXiv preprint arXiv:1605.07146, 2016.

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples

A. Local Intrinsic Dimensionality
Defense Details. The Local Intrinsic Dimensionality
(Amsaleg et al., 2015) “assesses the space-filling capability
of the region surrounding a reference example, based on the
distance distribution of the example to its neighbors” (Ma
et al., 2018). The authors present evidence that the LID is
significantly larger for adversarial examples generated by
existing attacks than for normal images, and they construct a
classifier that can distinguish these adversarial images from
normal images. Again, the authors indicate that LID is not
intended as a defense and only should be used to explore
properties of adversarial examples. However, it would be
natural to wonder whether it would be effective as a defense,
so we study its robustness; our results confirm that it is
not adequate as a defense. The method used to compute
the LID relies on finding the k nearest neighbors, a nondifferentiable operation, rendering gradient descent based
methods ineffective.
Let S be a mini-batch of N clean examples. Let ri (x)
denote the distance (under metric d(x, y)) between sample
x and its i-th nearest neighbor in S (under metric d). Then
LID can be approximated by
LIDd (x) = −

k
ri (x)
1X
log
k i=1
rk (x)

!−1

where k is a defense hyperparameter the controls the number of nearest neighbors to consider. The authors use the
distance function
dj (x, y) = f 1..j (x) − f 1..j (y)

2

to measure the distance between the jth activation layers.
The authors compute a vector of LID values for each sample:
−−→
LID(x) = {LIDdj (x)}nj=1 .
−−→
Finally, they compute the LID(x) over the training data
and adversarial examples generated on the training data,
and train a logistic regression classifier to detect adversarial
examples. We are grateful to the authors for releasing their
complete source code.

achieves 0% success. Because Carlini and Wagner’s `2
attack is unbounded, any time the attack does not reach
100% success indicates that the attack became stuck in a
local minima. When this happens, it is often possible to
slightly modify the loss function and return to 100% attack
success (Carlini & Wagner, 2017b).
In this case, we observe the reason that performing this
type of adaptive attack fails is that gradient descent does
not succeed in optimizing the LID loss, even though the
LID computation is differentiable. Computing the LID term
involves computing the k-nearest neighbors when computing ri (x). Minimizing the gradient of the distance to the
current k-nearest neighbors is not representative of the true
direction to travel in for the optimal set of k-nearest neighbors. As a consequence, we find that adversarial examples
generated with gradient methods when penalizing for a high
LID either (a) are not adversarial; or (b) are detected as
adversarial, despite penalizing for the LID loss.
Evaluation. We now evaluate what would happen if a
defense would directly apply LID to detect adversarial examples. Instead of performing gradient descent over a term
that is difficult to differentiate through, we have found that
generating high confidence adversarial examples (Carlini &
Wagner, 2017a) (completely oblivious to to the detector) is
sufficient to fool this detector. We obtain from the authors
their detector trained on both the Carlini and Wagner’s `2
attack and train our own on the Fast Gradient Sign attack,
both of which were found to be effective at detecting adversarial examples generated by other methods. By generating
high-confidence adversarial examples minimizing `∞ distortion, we are able to reduce model accuracy to 2% success
within  = 0.015. LID reports these adversarial examples
are benign at a 97% rate (unmodified test data is flagged as
benign with a 98% rate).
This evaluation demonstrates that the LID metric can be
circumvented, and future work should carefully evaluate if
building a detector relying on LID is robust to adversarial
examples explicitly targeting such a detector. This work
also raises questions whether a large LID is a fundamental
characteristic of all adversarial examples, or whether it is a
by-product of certain attacks.

Discussion. While LID is not a defense itself, the authors
assess the ability of LID to detect different types of attacks.

B. Defense-GAN

Through solving the formulation

Defense Details. The defender first trains a Generative
Adversarial Network with a generator G(z) that maps samples from a latent space (typically z ∼ N (0, 1)) to images
that look like training data. Defense-GAN takes a trained
classifier f (·), and to classify an input x, instead of returning f (x), returns f (arg minz |G(z) − x|). To perform this
projection to the manifold, the authors take many steps of
gradient descent starting from different random initializa-

min. |x − x0 |22 + α (`(x0 ) + LID-loss(x0 ))
the authors attempt to determine if the LID metric is a good
metric for detecting adversarial examples. Here, LID-loss(·)
is a function that can be minimized to reduce the LID score.
However, the authors report that this modified attack still

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples

tions.
Defense-GAN was not shown to be effective on CIFAR-10.
We therefore evaluate it on MNIST (where it was argued to
be secure).
Discussion. In Samangouei et al. (2018), the authors construct a white-box attack by unrolling the gradient descent
used during classification. Despite an unbounded `2 perturbation size, Carlini and Wagner’s attack only reaches
30% misclassification rate on the most vulnerable model
and under 5% on the strongest. This leads us to believe that
unrolling gradient descent breaks gradients.
Evaluation. We find that adversarial examples do exist
on the data manifold as described by the generator G(·).
However, Defense-GAN does not completely project to
the projection of the generator, and therefore often does not
identify these adversarial examples actually on the manifold.
We therefore present two evaluations. In the first, we assume that Defense-GAN were to able to perfectly project
to the data manifold, and give a construction for generating
adversarial examples. In the second, we take the actual implementation of Defense-GAN as it is, and perform BPDA
to generate adversarial examples with 50% success under
reasonable `2 bounds.
Evaluation A. Performing the manifold projection is nontrivial as an inner optimization step when generating adversarial examples. To sidestep this difficulty, we show that
adversarial examples exist directly on the projection of the
generator. That is, we construct an adversarial example
x0 = G(z ∗ ) so that |x − x0 | is small and c(x) 6= c(x0 ).
To do this, we solve the re-parameterized formulation
min. kG(z) − xk22 + c · `(G(z)).
We initialize z = arg minz |G(z) − x| (also found via gradient descent). We train a WGAN using the code the authors
provide (Gulrajani et al., 2017), and a MNIST CNN to
99.3% accuracy.
We run for 50k iterations of gradient descent for generating
each adversarial example; this takes under one minute per
instance. The unsecured classifier requires a mean `2 distortion of 0.0019 (per-pixel normalized, 1.45 un-normalized)
to fool. When we mount our attack, we require a mean
distortion of 0.0027, an increase in distortion of 1.46×; see
Figure 2 for examples of adversarial examples. The reason
our attacks succeed with 100% success without suffering
from vanishing or exploding gradients is that our gradient
computation only needs to differentiate through the generator G(·) once.

Figure 2. Images on the MNIST test set. Row 1: Clean images.
Row 2: Adversarial examples on an unsecured classifier. Row 3:
Adversarial examples on Defense-GAN.

Concurrent to our work, Ilyas et al. (2017) also develop a
nearly identical approach to Defense-GAN; they also find
it is vulnerable to the attack we outline above, but increase
the robustness further with adversarial training. We do not
evaluate this extended approach.
Evaluation B. The above attack does not succeed on
Defense-GAN. While the adversarial examples are directly
on the projection of the Generator, the projection process
will actually move it off the projection.
To mount an attack on the approximate projection process,
we use the BPDA attack regularized for `2 distortion. Our
attack approach is identical to that of PixelDefend, except
we replace the manifold projection with a PixelCNN with
the manifold projection by gradient descent on the GAN.
Under these settings, we succeed at reducing model accuracy to 55% with a maximum normalized distortion of .0051
for successful attacks.

