A Rotation and a Translation Suffice:
Fooling CNNs with Simple Transformations

1
Logan Engstrom * 1 Brandon Tran * 1 Dimitris Tsipras * 1 Ludwig Schmidt 1 Aleksander Madry
˛

arXiv:1712.02779v3 [cs.LG] 13 Feb 2018

Abstract
We show that simple transformations, namely
translations and rotations alone, are sufficient to
fool neural network-based vision models on a significant fraction of inputs. This is in sharp contrast
to previous work that relied on more complicated
optimization approaches that are unlikely to appear outside of a truly adversarial setting. Moreover, fooling rotations and translations are easy to
find and require only a few black-box queries to
the target model. Overall, our findings emphasize
the need for designing robust classifiers even in
natural, benign contexts.

1. Introduction
Neural network models are now widely embraced as cuttingedge solutions in computer vision (Krizhevsky et al., 2012;
He et al., 2016), speech recognition (Graves et al., 2013),
and text analysis (Collobert & Weston, 2008). However,
despite the near, and sometimes beyond, human-like results
they achieve on key benchmarks (He et al., 2015; Taigman
et al., 2014), we still do not understand the true reliability
and robustness of this performance. A prominent issue in
this context is the existence of so-called adversarial examples (Szegedy et al., 2013; Goodfellow et al., 2014), i.e.,
inputs that are almost indistinguishable from natural data to
the human eye, yet cause state-of-the-art classifiers to make
incorrect predictions with high confidence. This raises concerns about the use of neural networks in contexts where
reliability, dependability and security matter.

in various settings. However these methods are quite sophisticated and the resulting perturbations tend to be fairly
contrived. One might thus suspect that adversarial examples
constitute a problem only in a fully adversarial setting and
are unlikely to arise in more benign settings. This raises a
natural question:
Are neural network classifiers robust to simple,
naturally-occurring transformations of their input?
In this work, we address this question by studying two basic
image transformations: translations and rotations. While
these transformations appear natural to a human, we show
that they alone (i.e., without any additional perturbation
added) cause a significant drop in the model’s performance.
This remains to be the case even when the model has been
trained using appropriate data augmentation and no visual
information is lost due to these transformations (e.g. due to
cropping). See Figure 1 for an illustration.
Natural

Adversarial

“revolver”

“mousetrap”

“vulture”

“orangutan”

There is a long line of work (Szegedy et al., 2013; Goodfellow et al., 2014; Kurakin et al., 2016a;b; Sharif et al., 2016;
Moosavi-Dezfooli et al., 2016; Carlini & Wagner, 2016;
Papernot et al., 2017; Madry et al., 2017; Athalye et al.,
2017) on methods for constructing adversarial perturbations
*

Equal contribution 1 EECS, MIT, Massachusetts, USA. Correspondence to: Logan Engstrom <engstrom@mit.edu>, Brandon
Tran <btran115@mit.edu>, Dimitris Tsipras <tsipras@mit.edu>,
Ludwig Schmidt <ludwigs@mit.edu>, Aleksander Madry
˛
<madry@mit.edu>.

Figure 1: Examples of adversarial transformations and their
predictions in the standard and "black canvas" setting.

A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations

1.1. Our Methodology and Results
We start with standard models for the MNIST (LeCun et al.,
1998), CIFAR10 (Krizhevsky & Hinton, 2009), and ImageNet (Russakovsky et al., 2015) datasets. These models
achieve close to state-of-the-art performance on the respective benchmarks. We demonstrate however that even small
rotations and translations alone can cause these models to
have a significant drop in classification accuracy. This drop
ranges from 34% to as high as 90%.
Additionally, we show that one does not even need direct
access to the model (or its surrogate) to find such misclassifying transformations. In fact, choosing the worst out
of 10 random transformations is sufficient to reduce the
accuracy of these models by 26% on MNIST, 72% on CIFAR10, and 28% on ImageNet (Top 1). This demonstrates
that the existence of adversarial examples and the robustness of our neural network–based models should also be a
concern outside of safety/security contexts. We thus need to
properly examine and mitigate it in settings that are natural
and benign.
To this end, we examine possible ways to alleviate these
vulnerabilities. A natural first step here is to augment the
training procedure with rotations and translations. While
this does (largely) mitigate the problem on MNIST, the
models trained on CIFAR10 and ImageNet are still far from
being robust. We thus propose two natural methods for
further increasing the robustness of these models. These
methods are based on robust optimization and aggregation
of random input transformations. They end up offering significant improvement, but at the expense of a considerable
computational overhead. Even then, they are still not sufficient to completely mitigate the vulnerability. This suggests
that obtaining models robust to spatial transformations of
their inputs remains a challenge.
Finally, we examine the interplay between this variant of
adversarial perturbations and the widely used `∞ -based
variant. We observe that these two variants seem orthogonal
to each other. In particular, pixel-based robustness does
not imply spatial robustness, while combining spatial and
`∞ -bounded transformations seems to have a cumulative
effect. This emphasizes the need for our current, pixelwise notions of similarity used in the context of adversarial
examples to be significantly broadened and include other
natural spatial transformations, such as the rotations and
translations considered here.

• A simple attack based purely on rotations and translations is very effective against state-of-the-art neural
network models. This remains to be the case even
when the model has been trained with appropriate data
augmentation and no image information is lost during
the transformation.
• These attacks are fairly easy to execute. They require
only a few black-box queries to the model.
• One can take steps to increase the model’s robustness to
rotations and translations at the cost of increased training and/or inference runtime. However these methods
are still not sufficient to fully solve the problem.
• Robustness to `∞ -bounded perturbations does not
seem to affect spatial robustness. In fact, these two
notions seem to be orthogonal to each other.

2. Adversarial Rotations and Translations
Recall that, in the context of vision models, an adversarial
example for a given image x and a classifier C is an image
x0 that, on one hand, causes the classifier C to output a
different label on x0 than on x, i.e., to have C(x) 6= C(x0 ),
but, on the other hand, is “visually similar” to x. Clearly, the
notion of visual similarity is not precisely defined here. In
fact, providing such a precise and rigorous definition is extraordinarily difficult, as it would implicitly require formally
capturing the notion of human perception. Consequently,
previous work settled on assuming that x and x0 are close if
and only if kx − x0 kp ≤ ε for some p ∈ [0, ∞] and ε small
enough. Arguably, when two images are close in some `p
norm they are visually similar. However the converse is not
necessarily true. In this work, we aim to expand the range of
similarity measures considered to include for natural ones.
We will focus on rotations and translations which, while
changing most images significantly in all `p norms, tend to
not affect human perception.
Our initial goal is to develop sufficiently strong methods for
generating adversarial examples of this type. So far, in the
context of pixel-wise perturbations, the most successful approach for constructing adversarial examples has been to use
optimization methods on a suitable loss function (Szegedy
et al., 2013; Goodfellow et al., 2014; Carlini & Wagner,
2016). Following this approach, we parametrize our attack
method with a set of tunable parameters and then optimize
over these parameters. We perform this optimization in
three distinct ways:

1.2. Summary of Contributions
We perform extensive experiments that provide a finegrained understanding of the vulnerabilities described so
far on a wide spectrum of datasets and training regimes. In
summary, we show that:

• First-Order Method (FO): Starting from a random
choice of parameters, we iteratively take steps in the
direction of the loss function’s gradient. This is the direction that locally maximizes the loss of the classifier
(as a surrogate for misclassification probability). Note

A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations

that unlike the `p -norm case, we are not optimizing in
the pixel space but in the latent space of rotation and
translation parameters.
• Grid Search: We discretize the parameter space and
exhaustively examine every possible parametrization
of the attack to find one that causes the classifier to give
a wrong prediction (if such a parametrization exists).
Since our parameter space is small enough, this method
is computationally feasible (in contrast to a grid search
for `p -based adversaries).
• Worst-of-k: We randomly sample k different attack
parameter choices and choose the one on which the
model performs worse. As we increase k, this attack
interpolates between random choice and grid search.
Observe that while a first-order attack requires full knowledge of the model in order to compute the gradient of the
loss with respect to the input, the other two attacks do not.
They simply require evaluating a chosen input which can
be done even when there is only query access to the target
model.
We now need to define the exact range of attacks we want
to optimize over. For the case of rotation and translation
attacks, we wish to find parameters (δu, δv, θ) such that
rotating the original image θ degrees around the center and
then translating it by (δu, δv) pixels causes the classifier to
make a wrong prediction. Formally, the pixel at position
(u, v) is moved to the following position (assuming the point
(0, 0) is the center of the image):
 0 
    
u
cos θ − sin θ
u
δu
=
·
+
.
v0
sin θ
cos θ
v
δv
We implement this transformation in a differentiable manner using the spatial transformer blocks of (Jaderberg et al.,
2015). In order to deal with pixels mapped to non-integer
coordinates, these transformer units include a differentiable
bilinear interpolation routine. Since our loss function is differentiable with respect to the input and the transformation
is in turn differentiable with respect to its parameters, we
can apply a first-order optimization method to our problem.
By defining the spatial transformation for some x as
T (x; δu, δv, θ), we construct an adversarial perturbation
for x by solving the problem
max L(x0 , y),

δu,δv,θ

for x0 = T (x; δu, δv, θ) ,

(1)

where L is the loss function of the neural network1 , and y is
the correct label for example x. Since this is a non-concave
1
The loss L of the classifier is a function from images to real
numbers that expresses the performance of the network on the
particular example x (e.g., the cross-entropy between predicted
and correct distributions).

maximization problem, there are no formal guarantees for
the global optimality of our solution.

3. Improving Invariance to Spatial
Transformations
A natural idea to design models robust to rotations and
translations is to augment the training with such random
transformation. In order to further improve the model robustness, we propose two novel defense mechanism that
modify the training or inference method of the model.
Robust Optimization As a first step, instead of performing standard empirical risk minimization to train the model,
we will utilize robust optimization methods. Robust optimization has a rich history (Ben-Tal et al., 2009) and has
recently been applied successfully in the context of defending neural networks against adversarial examples (Madry
et al., 2017; Sinha et al., 2017; Raghunathan et al., 2018;
Kolter & Wong, 2017). The main barrier to applying robust optimization to improve spatial invariance is the lack
of an efficient procedure to compute the worst-case perturbation of a given example. Performing a grid search (as
described in Section 2), is prohibitive as this would increase
the training time by a factor close to the grid size. Moreover, the non-convexity of the landscape prevents first-order
methods from discovering such (approximately) worst-case
transformations efficiently (see Section 4 for details).
Given that we cannot fully optimize over the space of transformations, we use a coarse approximation provided by the
worst-of-10 adversary (described in Section 2). That is, first
we sample 10 random transformations of each training example uniformly from the space of allowed transformations.
We then evaluate the model on each of these transformations and choose to train on the one with highest loss. This
corresponds to minimizing a min-max formulation of the
problem similar to that in (Madry et al., 2017). Training
against such an adversary increases the training by a factor
of roughly six2 .
Aggregating Random Transformations For every example, most of the transformations don’t result in misclassification, as we will verify in Section 4. This motivates the
following inference procedure: compute a (small) number
of random transformations of the given image and output the
label that is predicted the most over these transformations
(majority vote). We constrain these random transformations
to be within 5% of the input image size in each translation
direction and up to 15 degrees of rotations. Observe that if
an adversary rotates an image by 30◦ (a valid attack in our
2

We need to perform 10 forward passes and one backwards pass
instead of one forward and one backwards required for standard
training.

A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations

threat model), we might end up evaluating the image on rotations of up to 45◦ . The training procedure and model can
remain unchanged while the inference time gets increased
by a small factor (equal to the number of transformations
we evaluate on).

parameter range. When rotating and translating the images,
we fill the empty space with zeros (black pixels).

Combining these methods These two methods are orthogonal and in some sense complementary. We can therefore combine robust training (using a worst-of-k adversary)
and majority inference to further increase the robustness of
our models.

• Standard natural training: out-of-the-box training procedure. For CIFAR10 this includes random left-right
flipping, random translations of ±4, and per image
standardization. For ImageNet, it includes random
left-right flipping, random cropping similar to that
in (Szegedy et al., 2015), and various color space transformations. We refer to this as the Standard model.

4. Experiments
4.1. Experimental Setup
We will consider standard architectures for the MNIST (LeCun et al., 1998), CIFAR10 (Krizhevsky & Hinton, 2009)
and ImageNet (Russakovsky et al., 2015) datasets. In order
to determine the extent to which misclassification is caused
by non-sufficient data augmentation during training, we will
examine various data augmentation methods.
Model Architecture For the MNIST dataset we use a
simple convolutional neural network derived from the TensorFlow Tutorial (tft). In order to obtain a fully convolutional version of the network, we replace the fully-connected
layer by two convolutional layers with 128 and 256 filters
each, followed by a global average pooling. For CIFAR10,
we consider a standard ResNet (He et al., 2016) model
with 4 groups of residual layers with filter sizes [16, 16,
32, 64] and 5 residual units each. We use naturally and
`∞ -adversarially trained models similar to these studied by
Madry et al. (Madry et al., 2017)3,4 . For ImageNet, we use a
ResNet-50 (He et al., 2016) architecture using the code from
the tensorpack repository (ten). We did not modify the
model architectures or training procedures. We will make
our full experimental code available soon.
Attack Space In order to maintain the visual similarity of
images to the natural ones we restrict the space of allowed
perturbations to be relatively small. We consider rotations of
at most 30 degrees and translations of at most (roughly) 10%
percent of the image size in each direction. This corresponds
to 3 pixels for MNIST (image size 28 × 28) and CIFAR10
(image size 32 × 32), and 24 pixels for ImageNet (image
size 299 × 299). For grid search attacks, we consider 5
values per translation direction and 31 values for rotations,
equally spaced. For first-order attacks, we use 200 steps
of Projected Gradient Descent of step size 0.01 times the
3
https://github.com/MadryLab/cifar10_
challenge
4
https://github.com/MadryLab/mnist_
challenge

Data Augmentation
for our models.

We consider five variants of training

• Standard `∞ -bounded adversarial training: The classifier is trained using `∞ -bounded adversarial examples computed using Projected Gradient Descent in the
pixel space at each step. For ImageNet there is not yet
a classifier that can be efficiently and reliably trained
against such an adversary. We refer to this model as
`∞ -Adversarially Trained.
• No random cropping: In order to examine the effects
of standard data augmentation during training we train
the CIFAR10 and ImageNet models without random
cropping. We still apply random left-right flips and the
rest of the transformations. We refer to this model as
No Crop.
• Random rotations and translations: We perform a uniformly random perturbation of training examples in the
attack space described above at each training step. That
is, on top of standard data augmentation, we translate
and rotate the image choosing parameters uniformly at
random from the allowed space. We refer to this model
as Augmentation 30.
• Random rotations and translations from larger intervals: Same as Augmentation 30, but we instead select
random perturbations from a superset of the attack
space. We sample random rotations of at most 40 degrees and translations of at most 4 pixels for MNIST
and CIFAR10, and 32 pixels for ImageNet. We refer
to this model as Augmentation 40.
4.2. Evaluating Model Robustness
We evaluate all models against random and grid search
adversaries (rotations and translations considered both separately and together) and report the results in Table 1. We visualize a random subset of successful attacks in Figures 4, 5,
and 6 of Appendix A.
Despite the high accuracy of standard models on natural
examples and their reasonable performance on random perturbations, a grid search adversary can significantly lower

A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations

the classifiers’ accuracy on the test set. For the standard
models, accuracy drops from 99% to 26% on MNIST, 93%
to 3% on CIFAR10, and 76% to 31% on ImageNet (Top 1
accuracy).
The addition of random rotations and translations during
training greatly improves both the random and adversarial
accuracy of the classifier for MNIST and CIFAR10, but less
so for ImageNet. Note that the Augmentation 40 model has
been trained on transformations of larger magnitude than
those that appear during the evaluation.
We perform a fine-grained investigation of our findings:
• For each dataset, we examine how many examples can
be fooled by rotations only, translations only, neither,
or both in Figure 2.
• We visualize the set of fooling angles for a random sample of the rotations-only grid in Figure 7 of Appendix A
and observe that they form a highly non-convex set.
• To investigate how brittle these networks are, we analyze the percentage of grid points fooling each example
in Figure 8 in the Appendix A. While the majority of
perturbations are benign, a significant fraction of them
fool each classifier.
Black Canvas Experiments One might wonder if the reduced accuracy of the models is due to the cropping applied
during the transformation. We verify that this is not the
case by padding each image in the CIFAR10 and ImageNet
datasets with zeros. This creates a "black canvas" version of
the dataset, ensuring that no information from the original
image is lost during translation and rotation. We show a
random set of adversarial examples in this case in Figure 10.
The results from a full evaluation are shown in Table 4.
4.3. Comparing Attack Methods
In Table 2 we compare different attack methods across models and datasets. We observe that worst-of-10 is a very
powerful adversary, despite the limited interaction it has
with the model. The first-order adversary performs significantly worse, and despite being better than random choice
fails to approximate the ground-truth accuracy of the models.

Understanding the Failure of First-Order Methods
The fact that first-order methods fail to reliably find adversarial examples is in sharp contrast to previous work (Carlini & Wagner, 2016; Madry et al., 2017). They find that,
for the case of pixel-based perturbations, the optimization
landscape is well-behaved allowing first-order methods to
consistently find maxima of good value. In the case of spatial perturbations, we observe that the non-concavity of the

problem is a significant barrier. We investigate this issue by
visualizing the loss landscape. For a few random examples
we plot the cross-entropy loss of the examples as a function
of translation and rotation. In Figure 3 we show one example for each dataset. Additional examples are visualized in
Figure 9 of the Appendix. We clearly observe that the loss
landscape is highly non-concave and is full of low-value
local maxima. It appears that the low-dimensionality of the
problem makes non-concavity a crucial obstacle. Even for
the case of MNIST where we observe fewer local maxima,
the large flat regions would prevent first-order methods from
finding transformations of high loss.
Relation to Black-Box Attacks The performance of the
worst-of-10 adversary is impressive given the limited interaction with the model. The adversary performs 10 random,
non-adaptive queries to the model and is able to find adversarial examples for a large fraction of inputs. Recently,
there has been significant interest (Papernot et al., 2017;
Chen et al., 2017; Bhagoji et al., 2017; Ilyas et al., 2017)
in designing black-box attacks for neural network models.
These attacks rely only on black-box queries to the model,
without any additional information and aim to minimize the
number of such queries required. Our results propose a very
simple baseline strategy that produces adversarial examples
for a significant fraction of inputs with very few queries.
Combining Spatial and `∞ -bounded perturbations In
Table 1 we observe that models trained to be robust to `∞
attacks do not achieve higher robustness to spatial attacks.
This provides evidence that the two families of perturbation are orthogonal to each other. We further investigate
this possibility by considering a combined adversary that
utilizes `∞ bounded perturbations on top of rotations and
translations. The results are shown in Figure 11. We indeed observe that, when combined, these attack reduce the
classification accuracy in an additive manner.
4.4. Evaluating Our Defense Methods
In Table 1 we observe that training with a worst-of-10 adversary greatly increases the spatial robustness of the model. In
fact, it performs significantly better that simply augmenting
training with random transformations. We believe that using
more reliable methods to compute the worst-case transformations will further improve these results.
Our results for majority-based inference are presented in
Table 3. Again, we observe a significant increase in the
model’s performance in the natural and adversarial setting.
By combining these two defense, we manage to improve
the worst-case performance of the models from 26% to 98%
on MNIST, from 3% to 82% on CIFAR10, and from 31%
to 56% on ImageNet (Top 1).

A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations

Table 1: Accuracy of different models on the natural evaluation set and against rotation and translation adversaries on the
MNIST, CIFAR10, and ImageNet datasets. The allowed transformations are translations of (roughly) 10% of the image
pixels and ±30◦ rotation. (Image sizes are 28 × 28 for MNIST, 32 × 32 for CIFAR10, and 299 × 299 for ImageNet).
The attack parameters are chosen through random sampling or grid search with rotations and translations considered both
together and separate.
Model

Natural

Random

Grid

Rand. Trans.

Trans. Grid

Rand. Rot.

Rot. Grid

Standard

99.31%

94.23%

26.02%

98.61%

89.80%

95.68%

70.98%

`∞ -Adversarially Trained

93.72%

34.13%

95.27%

72.03%

99.53%

99.35%

95.79%

99.47%

98.66%

99.34%

98.23%

◦

Aug. 40 (±4±, ±40 )

99.34%

99.31%

96.95%

99.39%

98.65%

99.40%

98.49%

Worst-of-10 Training (30)

99.48%

99.37%

97.32%

99.50%

99.01%

99.39%

98.62%

Worst-of-10 Training (40)

99.42%

99.39%

97.88%

99.45%

98.89%

99.36%

98.85%

Standard

92.62%

60.93%

2.80%

88.54%

66.17%

75.36%

24.71%

No Crop

90.34%

54.64%

1.86%

81.95%

46.07%

69.23%

18.34%

80.21%

58.33%

6.02%

78.15%

59.02%

62.85%

20.98%

Aug. 30(±3px, ±30 )

90.02%

90.92%

58.90%

91.76%

79.01%

91.14%

76.33%

Aug. 40 (±4px, ±40◦ )

88.83%

91.18%

61.69%

91.53%

77.42%

91.10%

76.80%

Worst-of-10 Training (30)

91.34%

92.35%

69.17%

92.43%

83.01%

92.33%

81.82%

Worst-of-10 Training (40)

91.00%

92.11%

71.15%

92.28%

82.15%

92.53%

82.25%

Standard

75.96%

63.39%

31.42%

73.24%

60.42%

67.90%

44.98%

No Crop

70.81%

59.09%

16.52%

66.75%

45.17%

62.78%

34.17%

65.96%

68.60%

32.90%

70.27%

45.72%

69.28%

47.25%

Aug. 40 (±32px, ±40 )

66.19%

67.58%

33.86%

69.50%

44.60%

68.88%

48.72%

Worst-of-10 Training (30)

76.14%

73.19%

52.76%

74.42%

61.18%

73.74%

61.06%

Worst-of-10 Training (40)

74.64%

71.36%

50.23%

72.86%

59.34%

71.95%

59.23%

CIFAR10
ImageNet
Percent of Test Set Fooled

Aug. 30 (±3px, ±30 )

`∞ -Adversarially Trained
◦

Aug. 30 (±24px, ±30◦ )
◦

100
80
60
40
20
0

Stand

A
D
D
ard dv. Traine ata Aug. ata Aug.
30
40
d

MNIST

80
60
40
20
0

Stand Adv. No C Data Data
Aug. Aug.
ard Traine rop
30
40
d

CIFAR10

Percent of Test Set Fooled

1.20%

Percent of Test Set Fooled

88.02%

MNIST

98.65%

◦

60

Any
Only Trans
Only Rot
Both
None

40
20
0

Stand

N
D
D
ard o Crop ata Aug. ata Aug.
30
40

ImageNet

Figure 2: Fine-grained dataset analysis. For each model, we visualize what percent of the test set can be fooled via various
methods. We compute how many examples can be fooled with either translations or rotations ("any"), how many can be
fooled only by one of these, and how many require a combination to be fooled ("both").

A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations

Model

Natural

Worst-of-10

FO

Grid

Standard

99.31%

73.32%

79.84%

26.02%

`∞ -Adversarially Trained

98.65%

51.18%

81.23%

1.20%

◦

99.53%

98.33%

98.78%

95.79%

◦

Aug. 40 (±4±, ±40 )

99.34%

98.49%

98.74%

96.95%

Standard

92.62%

20.13%

62.69%

2.80%

No Crop

90.34%

15.04%

52.27%

1.86%

Aug. 30 (±3px, ±30 )

`∞ -Adversarially Trained

80.21%

19.38%

33.24%

6.02%

◦

90.02%

79.92%

85.92%

58.92%

◦

Aug. 40 (±4px, ±40 )

88.83%

80.47%

85.48%

61.69%

Standard

75.96%

47.83%

63.12%

31.42%

No Crop

70.81%

35.52%

55.93%

16.52%

65.96%

50.62%

66.05%

32.90%

66.19%

51.11%

66.14%

33.86%

Aug. 30 (±3px, ±30 )

Aug. 30 (±24px, ±30◦ )
◦

Aug. 40 (±32px, ±40 )

MNIST

CIFAR-10

ImageNet

10

2

3

2

0

0
30
20
10

1
10 ta
Trans 0 1
20 Ro
lation
2
LR
3 30

n
tio

7
6
5
4
3
2
1
0
30
20
10

2.5
2.0

Xent Loss

6
4

Xent Loss

8

3

2

0

tio

10 ta
Trans 0 1
20 Ro
lation
2
LR
3 30
1

n

Xent Loss

ImageNet

CIFAR10

MNIST

Table 2: Comparison of attack methods across datasets and models. Worst-of-10 is very effective and significantly reduces
the model accuracy despite the limited interaction. The first-order (FO) adversary performs poorly, despite the large number
of steps allowed.

1.5
1.0
0.5

20

0.0
30
20
10

10

Trans 0 10
lation
LR 20

30

0 on
10 tati
20 Ro

Figure 3: Loss landscape of a random example for each dataset when performing left-right translations and rotations.
Translations and rotations are restricted to 10% of the image pixels and 30 deg respectively. We observe that the landscape
is significantly non-concave, making rendering FO methods for adversarial example generation powerless. Additional
examples are visualized in Figure 9 of the Appendix.

A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations

Table 3: Majority Defense. Accuracy of different models on
the natural evaluation set and against a combined rotation
and translation adversary using aggregation of multiple random transformations. For each image, natural or perturbed,
10 random transformations are performed and the model
outputs a majority vote over predicted labels.
Natural Acc.

MNIST

Model

Stand.

Vote

Grid Acc.
Stand.

Vote

Standard

99.31% 98.71% 26.02% 18.80%

Aug 30.

99.53% 99.41% 95.79% 95.32%

Aug 40.

99.34% 99.25% 96.95% 97.65%

W-10 (30) 99.48% 99.40% 97.32% 96.95%

CIFAR10

W-10 (40) 99.42% 99.41% 97.88% 98.47%
Standard

92.62% 80.37%

2.82%

7.85%

Aug 30.

90.02% 92.70% 58.90% 69.65%

Aug 40.

88.83% 92.50% 61.69% 76.54%

W-10 (30) 91.34% 93.38% 69.17% 77.33%

ImageNet

W-10 (40) 91.00% 93.40% 71.15% 81.52%
Standard

75.96% 73.19% 31.42% 40.21%

Aug 30.

65.96% 72.44% 32.90% 44.46%

Aug 40.

66.19% 71.46% 33.86% 46.98%

W-10 (30) 76.14% 74.92% 52.76% 56.45%
W-10 (40) 74.64% 73.38% 50.23% 56.23%

5. Related Work
The fact that small rotations and translation can fool neural
network-based classifiers on MNIST and CIFAR10 was first
observed in (Fawzi & Frossard, 2015). The main difference
from our work is that they focus on finding the minimum
transformation required to fool a classifier. They use this as a
measure for qualitative comparison of different architectures
and training procedures. In contrast, we fix a range of small
transformations, and we study the robustness of a model
within this space.
Recently, (Xiao et al., 2018) and (Tramèr & Boneh, 2017)
observed independently that it is possible to use various
spatial transformations to construct adversarial examples
for naturally and adversarially trained models. The main
difference from our work is that we show even very simple
transformations (translations and rotations) are sufficient
to break a variety of classifiers, while the transformations
employed in (Xiao et al., 2018) and (Tramèr & Boneh, 2017)
are more involved. The transformation in (Xiao et al., 2018)
is based on performing a displacement of individual pixels in

the original image that is constrained to be globally smooth
and then optimized for misclassification probability. In
(Tramèr & Boneh, 2017), one considers an `∞ –bounded
pixel-wise perturbation of a version of the original image
that has been slightly rotated and in which a few random
pixels have been flipped. Both these methods require direct
access to the attacked model (or its surrogate) to compute
(or at least estimate) the gradient of the loss function with
respect to the model’s input. In contrast, our attacks can
be implemented using just a small number of random, nonadaptive transformations of the input.

6. Conclusions
We examined the robustness of state-of-the-art vision classifiers to translations and rotations. We observed that even
a small number of randomly chosen perturbations of the
input are sufficient to considerably degrade the model performance.
The vulnerability of state-of-the-art neural networks to such
simple and naturally occurring spatial perturbations indicates that adversarial robustness should be a concern not
only in the truly adversarial setting. The finding that such
misclassified perturbations can be constructed by trying out
a small number of randomly chosen input transformations
emphasizes that such classifiers are indeed quite brittle. Additional techniques need to be incorporated in the architecture and training procedures of modern classifiers to achieve
worst-case translational and rotational robustness. Also,
these results underline the need for considering broader
notions of similarity than just pixel-wise distances when
studying adversarial misclassification attacks. In particular,
we view combining the pixel-wise distances with rotations
and translations as a next step towards capturing the “right”
notion of similarity in the context of images.
Additionally, we find that classifiers trained to be robust
against `∞ -bounded pixel-based perturbations do not exhibit any additional spatial robustness, suggesting that these
are orthogonal concepts. Combining `∞ -bounded attacks
and spatial transformations seems to have a cumulative effect on the model.
Finally, we explored methods to increase spatial robustness of vision models. Augmenting the training data with
random transformations increases the resulting classifier’s
robustness, but does not completely alleviate the issue. We
observed that randomly sampling a few transformations of
each training point and training on the worst significantly
improves the model’s performance. Furthermore, by making
predictions based on a majority vote of randomly perturbed
versions of the input, we can further increase classification
accuracy in the adversarial setting.

A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations

References
Tensorpack repository.
https://github.com/
ppwwyyxx/tensorpack.
TensorFlow tutorial: Deep MNIST for experts. URL
https://www.tensorflow.org/versions/
r0.12/tutorials/mnist/pros/.

He, Kaiming, Zhang, Xiangyu, Ren, Shaoqing, and Sun,
Jian. Deep residual learning for image recognition. In
Proceedings of the IEEE Conference on Computer Vision
and Pattern Recognition, pp. 770–778, 2016.
Ilyas, Andrew, Engstrom, Logan, Athalye, Anish, and Lin,
Jessy. Query-efficient black-box adversarial examples.
arXiv preprint arXiv:1712.07113, 2017.

Athalye, Anish, Engstrom, Logan, Ilyas, Andrew, and
Kwok, Kevin. Synthesizing robust adversarial examples.
arXiv preprint arXiv:1707.07397, 2017.

Jaderberg, Max, Simonyan, Karen, Zisserman, Andrew, et al.
Spatial transformer networks. In Advances in Neural
Information Processing Systems, pp. 2017–2025, 2015.

Ben-Tal, A., El Ghaoui, L., and Nemirovski, A.S. Robust
Optimization. Princeton University Press, 2009.

Kolter, J Zico and Wong, Eric. Provable defenses against
adversarial examples via the convex outer adversarial
polytope. arXiv preprint arXiv:1711.00851, 2017.

Bhagoji, Arjun Nitin, He, Warren, Li, Bo, and Song, Dawn.
Exploring the space of black-box attacks on deep neural
networks. arXiv preprint arXiv:1712.09491, 2017.
Carlini, Nicholas and Wagner, David. Towards evaluating the robustness of neural networks. arXiv preprint
arXiv:1608.04644, 2016.
Chen, Pin-Yu, Zhang, Huan, Sharma, Yash, Yi, Jinfeng,
and Hsieh, Cho-Jui. Zoo: Zeroth order optimization
based black-box attacks to deep neural networks without
training substitute models. In Proceedings of the 10th
ACM Workshop on Artificial Intelligence and Security, pp.
15–26. ACM, 2017.
Collobert, Ronan and Weston, Jason. A unified architecture
for natural language processing: Deep neural networks
with multitask learning. In Proceedings of the 25th international conference on Machine learning, pp. 160–167.
ACM, 2008.
Fawzi, Alhussein and Frossard, Pascal. Manitest: Are classifiers really invariant? In British Machine Vision Conference (BMVC), 2015.
Goodfellow, Ian J., Shlens, Jonathon, and Szegedy, Christian. Explaining and harnessing adversarial examples.
arXiv preprint arXiv:1412.6572, 2014.
Graves, Alex, Mohamed, Abdel-rahman, and Hinton, Geoffrey. Speech recognition with deep recurrent neural
networks. In Acoustics, speech and signal processing
(icassp), 2013 ieee international conference on, pp. 6645–
6649. IEEE, 2013.
He, Kaiming, Zhang, Xiangyu, Ren, Shaoqing, and Sun,
Jian. Delving deep into rectifiers: Surpassing human-level
performance on imagenet classification. In Proceedings
of the IEEE international conference on computer vision,
pp. 1026–1034, 2015.

Krizhevsky, Alex and Hinton, Geoffrey. Learning multiple
layers of features from tiny images. 2009.
Krizhevsky, Alex, Sutskever, Ilya, and Hinton, Geoffrey E.
Imagenet classification with deep convolutional neural
networks. In Advances in neural information processing
systems, pp. 1097–1105, 2012.
Kurakin, Alexey, Goodfellow, Ian, and Bengio, Samy. Adversarial examples in the physical world. arXiv preprint
arXiv:1607.02533, 2016a.
Kurakin, Alexey, Goodfellow, Ian J., and Bengio, Samy.
Adversarial machine learning at scale. arXiv preprint
arXiv:1611.01236, 2016b.
LeCun, Yann, Bottou, Léon, Bengio, Yoshua, and Haffner,
Patrick. Gradient-based learning applied to document
recognition. Proceedings of the IEEE, 86(11):2278–2324,
1998.
Madry, Aleksander, Makelov, Aleksandar, Schmidt, Ludwig,
Tsipras, Dimitris, and Vladu, Adrian. Towards deep
learning models resistant to adversarial attacks. arXiv
preprint arXiv:1706.06083, 2017.
Moosavi-Dezfooli, Seyed-Mohsen, Fawzi, Alhussein, and
Frossard, Pascal. Deepfool: A simple and accurate
method to fool deep neural networks. In 2016 IEEE
Conference on Computer Vision and Pattern Recognition,
CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pp.
2574–2582, 2016.
Papernot, Nicolas, McDaniel, Patrick, Goodfellow, Ian, Jha,
Somesh, Celik, Z Berkay, and Swami, Ananthram. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, pp. 506–519. ACM,
2017.

A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations

Raghunathan, Aditi, Steinhardt, Jacob, and Liang, Percy.
Certified defenses against adversarial examples. International Conference on Learning Representations,
2018. URL https://openreview.net/forum?
id=Bys4ob-Rb.
Russakovsky, Olga, Deng, Jia, Su, Hao, Krause, Jonathan,
Satheesh, Sanjeev, Ma, Sean, Huang, Zhiheng, Karpathy, Andrej, Khosla, Aditya, Bernstein, Michael, Berg,
Alexander C., and Fei-Fei, Li. ImageNet Large Scale
Visual Recognition Challenge. International Journal of
Computer Vision (IJCV), 115(3):211–252, 2015. doi:
10.1007/s11263-015-0816-y.
Sharif, Mahmood, Bhagavatula, Sruti, Bauer, Lujo, and
Reiter, Michael K. Accessorize to a crime: Real and
stealthy attacks on state-of-the-art face recognition. In
Proceedings of the 2016 ACM SIGSAC Conference on
Computer and Communications Security, Vienna, Austria,
October 24-28, 2016, pp. 1528–1540, 2016.
Sinha, Aman, Namkoong, Hongseok, and Duchi, John. Certifiable distributional robustness with principled adversarial training. arXiv preprint arXiv:1710.10571, 2017.
Szegedy, Christian, Zaremba, Wojciech, Sutskever, Ilya,
Bruna, Joan, Erhan, Dumitru, Goodfellow, Ian J., and
Fergus, Rob. Intriguing properties of neural networks.
arXiv preprint arXiv:1312.6199, 2013.
Szegedy, Christian, Liu, Wei, Jia, Yangqing, Sermanet,
Pierre, Reed, Scott, Anguelov, Dragomir, Erhan, Dumitru,
Vanhoucke, Vincent, and Rabinovich, Andrew. Going
deeper with convolutions. In Proceedings of the IEEE
conference on computer vision and pattern recognition,
pp. 1–9, 2015.
Taigman, Yaniv, Yang, Ming, Ranzato, Marc’Aurelio, and
Wolf, Lior. Deepface: Closing the gap to human-level performance in face verification. In Proceedings of the IEEE
conference on computer vision and pattern recognition,
pp. 1701–1708, 2014.
Tramèr, Florian and Boneh, Dan. Personal communication,
2017.
Xiao, Chaowei, Zhu, Jun-Yan, Li, Bo, He, Warren, Liu,
Mingyan, and Song, Dawn. Spatially transformed adversarial examples. International Conference on Learning
Representations, 2018. URL https://openreview.
net/forum?id=HyydRMZC-.

A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations

A. Omitted Tables and Figures
Original
7

Standard
Perturbed
9

Adv. Trained
Original
Perturbed
7

9

Data Aug. 30
Original
Perturbed
9

2

Data Aug. 40
Original
Perturbed
5

6

2

9

2

7

5

6

3

8

1

7

1

7

3

8

2

1

0

5

0

5

6

4

9

4

4

0

4

0

4

2

4

8

1

7

1

7

7

2

7

1

4

9

4

9

7

2

4

9

9

6

9

2

2

1

2

7

5

6

5

6

3

7

2

9

Figure 4: MNIST. Successful adversarial examples for the models studied in Section 4. Rotations are restricted to be within
30◦ of the original image and translations up to 3 pixels per direction (image size 28 × 28). Each example is visualized
along with its predicted label in the original and perturbed versions.

A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations

Original
ship

Standard
Perturbed

automobile

Adv. Trained
Original
Perturbed
airplane

ship

Data Aug. 30
Original
Perturbed
ship

automobile

Data Aug. 40
Original
Perturbed
ship

automobile

ship

dog

automobile

truck

airplane

automobile

airplane

automobile

frog

dog

frog

bird

frog

dog

frog

cat

frog

bird

automobile

dog

automobile

truck

automobile

cat

automobile

dog

airplane

dog

dog

cat

automobile

frog

truck

cat

truck

frog

ship

airplane

airplane

dog

ship

airplane

horse

cat

dog

cat

dog

horse

ship

cat

ship

automobile

horse

truck

ship

airplane

airplane

bird

airplane

bird

horse

deer

horse

dog

Figure 5: CIFAR10. Successful adversarial examples for the models studied in Section 4. Rotations are restricted to be
within 30◦ of the original and translations up to 3 pixels per directions (image size 32 × 32). Each example is visualized
along with its predicted label in the original and perturbed version.

A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations

Original

Standard
Perturbed

Adv. Trained
Original
Perturbed

Data Aug. 30
Original
Perturbed

Data Aug. 40
Original
Perturbed

Petri dish

tray

Petri dish

bell pepper

Petri dish

tray

Petri dish

cucumber

boathouse

guillotine

boathouse

umbrella

boathouse

guillotine

boathouse

plastic bag

totem pole

wallet

wallet

envelope

wallet

envelope

totem pole

swimming trunks

golfcart

trailer truck

goblet

red wine

golfcart

car mirror

wallet

refrigerator

goblet

red wine

toyshop

perfume

goblet

red wine

capuchin

titi

toyshop

confectionery

garter snake

alligator lizard

garter snake

African crocodile

golfcart

minivan

garter snake

African crocodile

hyena

brown bear

hyena

platypus

goblet

red wine

hyena

cheetah

china cabinet

thimble

damselfly

dragonfly

toyshop

face powder

damselfly

dragonfly

barrow

barrel

china cabinet

golf ball

water snake

night snake

Figure 6: ImageNet. Successful adversarial examples for the models studied in Section 4. Rotations are restricted to be
within 30◦ of the original and translations up to 24 pixels per directions (image size 299 × 299). Each example is visualized
along with its predicted label in the original and perturbed version.

A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations

MNIST

Adv. Trained

Data Aug. 30

Data Aug. 40

Examples

Standard

-30

-15

0
Angle

15

30 -30

0
Angle

15

30 -30

CIFAR10

Adv. Trained

-15

0
Angle

15

30 -30

Data Aug. 30

-15

0
Angle

15

30

Data Aug. 40

Examples

Standard

-15

-30

-15

0
Angle

15

30 -30

-15

15

30 -30

ImageNet

No Crop

-15

0
Angle

15

30 -30

Data Aug. 30

-15

0
Angle

15

30

Data Aug. 40

Examples

Standard

0
Angle

-30

-15

0
Angle

15

30 -30

-15

0
Angle

15

30 -30

-15

0
Angle

15

30 -30

-15

0
Angle

15

30

Figure 7: Visualizing which angles fool the classifier for 50 random examples. For each dataset and model, we visualize one
example per row. Red corresponds to misclassification of the images. We observe that the angles fooling the models form a
highly non-convex set.

A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations

% of originally correct
examples misclassified

100

100

80

80

60

60

60

40

40

40

20

20

20

100
% of originally correct
examples misclassified

100

MNIST
Translations Only

80

0 0.0

0.2 0.4 0.6 0.8 1.0
Fraction of transformations
fooling the model
Translations and Rotations

0 0.0

100

0.2 0.4 0.6 0.8 1.0
Fraction of transformations
fooling the model

CIFAR10
Translations Only

0 0.0

100

80

80

80

60

60

60

40

40

40

20

20

20

0 0.0

80
% of originally correct
examples misclassified

Translations and Rotations

0.2 0.4 0.6 0.8 1.0
Fraction of transformations
fooling the model
Translations and Rotations

0 0.0

80

0.2 0.4 0.6 0.8 1.0
Fraction of transformations
fooling the model

ImageNet
Translations Only

0 0.0

80

60

60

60

40

40

40

20

20

20

0 0.0

0.2 0.4 0.6 0.8
Fraction of transformations
fooling the model

1.0

0 0.0

0.2 0.4 0.6 0.8
Fraction of transformations
fooling the model

1.0

0 0.0

Rotations Only
Standard
Adv. Trained
Data Aug. 30
Data Aug. 40

0.2 0.4 0.6 0.8 1.0
Fraction of transformations
fooling the model
Rotations Only
Standard
Adv. Trained
No Crop
Data Aug. 30
Data Aug. 40

0.2 0.4 0.6 0.8 1.0
Fraction of transformations
fooling the model
Rotations Only
Standard
No Crop
Data Aug. 30
Data Aug. 40

0.2 0.4 0.6 0.8
Fraction of transformations
fooling the model

1.0

Figure 8: Cumulative Density Function plots. For each fraction of grid points p, we plot the percentage of correctly classified
test set examples that are fooled by at least p of the grid points. For instance, we can see from the first plot, MNIST
Translations and Rotations, that approximately 10% of the correctly classified natural examples are misclassified under 1/5
of the grid points transformations.

A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations

ImageNet

Xent Loss

tio

Ro
ta

Xent Loss

tat

ion

0
10

1e 1

Xent Loss

1.4
1.2
1.0
0.8
0.6
0.4
0.2
0.0
30
20
10

20

10

Trans 0 10
lation
LR 20

30

20

0
10

Xent Loss

4

0

2
1

20

10

Trans 0 10
lation
LR 20

0
10

0
30
20
10

tio
n

1

3

30

Ro
ta

2

10
Trans 0 1
20
lation
2
30
LR
3

tio
n

3

Xent Loss

tat
i
1.6
1.4
1.2
1.0
0.8
0.6
0.4
0.2
0.0
30
20
10

Ro
ta

Ro
ta

tio
n

0

0
30
20
10

30

20

tat
i

0

Xent Loss

2

1

8
7
6
5
4
3
2
1
0
30
20
10

10

Trans 0 10
lation
LR 20

1.0
0.5
0.0
30
20
10

Ro

1

1

2

20

Xent Loss

2

10
Trans 0 1
20
lation
LR 2 3 30

3

10
Trans 0 1
20
lation
2
30
LR
3

1.5

Ro

tat
i

3

4

3

2.5

Ro

0

1e 3

0

0
10

on

1

Xent Loss

1

on

2

10
Trans 0 1
20
lation
LR 2 3 30

Ro

3

30

20

Xent Loss

tat

2

10
Trans 0 1
20
lation
2
30
LR
3

ion

3

2.00
1.75
1.50
1.25
1.00
0.75
0.50
0.25
0.00
30
20
10

10

Trans 0 10
lation
LR 20

2.0

tat

0

20

Ro

1

ion

2

10
Trans 0 1
20
lation
2
30
LR
3

Ro

3

20.0
17.5
15.0
12.5
10.0
7.5
5.0
2.5
0.0
30
20
10

3.5
3.0
2.5
2.0
1.5
1.0
0.5
0.0
30
20
10

n

Xent Loss

0

n

1

Xent Loss

14
12
10
8
6
4
2
0
30
20
10

2

10
Trans 0 1
20
lation
LR 2 3 30

tio

3

Ro
ta

0

8
7
6
5
4
3
2
1
0
30
20
10

on

1

tio

2

10
Trans 0 1
20
lation
LR 2 3 30

Ro
ta

3

n

Xent Loss

1.6
1.4
1.2
1.0
0.8
0.6
0.4
0.2
0.0
30
20
10

1e 3

CIFAR-10
1e 3

MNIST

20

Figure 9: Loss landscape of 4 random examples for each dataset when performing left-right translations and rotations.
Translations and rotations are restricted to 10% of the image pixels and 30◦ respectively. We observe that the landscape is
significantly non-concave, making rendering FO methods for adversarial example generation powerless.

A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations

Original
airplane

Cifar

Perturbed

ImageNet
Original
Perturbed

ship

harvester

screen

automobile

horse

vulture

orangutan

airplane

dog

sea snake

slug

dog

cat

crane

pier

horse

dog

soup bowl

eggnog

airplane

bird

bakery

packet

Figure 10: Sample adversarial transformations for the "black-canvas" setting for the standard models on CIFAR10 and
ImageNet.

A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations

Table 4: Evaluation of a subset of Table 1 in the “black-canvas” setting (images are zero-padded to avoid cropping due to
rotations and translations). The models are trained on padded images.
Natural

Random

Worst-of-10

Grid

Trans. Grid

Rot. Grid

91.81%

70.23%

25.51%

6.55%

83.38%

12.44%

89.70%

52.86%

14.14%

1.17%

47.94%

9.46%

Aug. 30 (±3px, ±30 )

91.45%

90.82%

80.53%

63.64%

82.28%

76.32%

Aug. 40 (±4px, ±40◦ )

91.24%

91.00%

81.81%

66.64%

81.75%

78.57%

Standard

73.60%

46.59%

29.51%

15.38%

28.03%

23.81%

66.28%

38.70%

14.17%

3.43%

8.87%

10.97%

Aug. 30 (±24px, ±30 )

64.60%

67.75%

47.32%

28.51%

45.33%

39.33%

Aug. 40 (±32px, ±40◦ )

49.20%

57.69%

38.36%

22.10%

32.84%

32.95%

ImageNet

CIFAR10

Standard
No Crop
◦

No Crop
◦

MNIST

Accuracy

Standard

Adv. Trained

Data Aug. 30

100
80
60
40
20
0

100
80
60
40
20
0
0

0.05

ε

0.1

0

0.15

Data Aug. 40

0.05

ε

0.1

0.15

`∞
random + `∞
grid + `∞

100
80
60
40
20
0

100
80
60
40
20
0
0

0.05

ε

0.1

0.15

0

0.05

ε

0.1

0.15

CIFAR10

Accuracy

Standard

Adv. Trained

100
80
60
40
20
0

Data Aug. 30

100
80
60
40
20
0
0

0.5

1
ε

1.5

2

Data Aug. 40

100
80
60
40
20
0
0

0.5

1
ε

1.5

2

`∞
random + `∞
grid + `∞

100
80
60
40
20
0
0

0.5

1
ε

1.5

2

0

0.5

1
ε

1.5

2

ImageNet

Accuracy

Standard
100
80
60
40
20
0

No Crop
100
80
60
40
20
0

0 0.1 0.2 0.3 0.4 0.5
ε

Data Aug. 30
100
80
60
40
20
0

0 0.1 0.2 0.3 0.4 0.5
ε

Data Aug. 40
100
80
60
40
20
0

0 0.1 0.2 0.3 0.4 0.5
ε

`∞
random + `∞
grid + `∞

0 0.1 0.2 0.3 0.4 0.5
ε

Figure 11: Accuracy of different classifiers against `∞ -bounded adversaries with various values of ε and spatial transformations. For each value of ε, we perform PGD to find the most adversarial `∞ -bounded perturbation. Additionally, we
combine PGD with random rotations and translations and with a grid search over rotations and translations in order to find
the transformation that combines with PGD in the most adversarial way.

