Disentangled Deep Autoencoding Regularization for Robust Image Classification
Zhenyu Duan Martin Renqiang Min Li Erran Li Mingbo Cai Yi Xu Bingbing Ni


Distributed Byzantine Tolerant Stochastic Gradient Descent in the Era of Big Data
    Richeng Jin
Department of ECE 

North Carolina State University

rjin2@ncsu.edu
Xiaofan He
Electronic Information School 

Wuhan University

xiaofanhe@whu.edu.cn
Huaiyu Dai
Department of ECE 

North Carolina State University

hdai@ncsu.edu
    December 30, 2023
===================================================================================================================================================================================================================================================




  In spite of achieving revolutionary successes in machine learning, deep convolutional neural networks have been recently found to be vulnerable to adversarial attacks and difficult to generalize to novel test images with reasonably large geometric transformations. Inspired by a recent neuroscience discovery revealing that primate brain employs disentangled shape and appearance representations for object recognition, we propose a general disentangled deep autoencoding regularization framework that can be easily applied to any deep embedding based classification model for improving the robustness of deep neural networks. Our framework effectively learns disentangled appearance code and geometric code for robust image classification, which is the first disentangling based method defending against adversarial attacks and complementary to standard defense methods. Extensive experiments on several benchmark datasets show that, our proposed regularization framework leveraging disentangled embedding significantly outperforms traditional unreguarlized convolutional neural networks for image classification on robustness against adversarial attacks and generalization to novel test data.




§ INTRODUCTION

Deep convolutional neural networks (CNNs) have achieved revolutionary successes in many machine learning tasks such as image classification <cit.>, object detection <cit.>, image segmentation <cit.>, image captioning <cit.>, video captioning <cit.>, text-to-video synthesis <cit.>, etc. In spite of these successes, CNNs have recently been shown to be vulnerable to adversarial images which are created by adding tiny perturbations to legitimate input images <cit.>. These adversarial images that are indistinguishable from real input images by human observers can cause CNNs to output any target prediction that the attacker wants to have, suggesting that the representations learned by CNNs still lack the robustness and the capability of generalization that biological neural systems exhibit. This security issue of CNNs poses serious challenges in applying deep learning to mission-critical applications such as autonomous driving.

Various efforts have been made recently to improve the situation. Theoretical approaches such as  <cit.> propose to learn deep ReLU-based classifiers that are provably robust against norm-bounded adversarial perturbations on the training data. However, the method still can not scale to realistic network sizes. Several empirical defense methods have been proposed <cit.>. Nevertheless, as shown in  <cit.>, many of these approaches do not have guaranteed defenses. On the other hand, neuroscience inspired approaches such as  <cit.> leverage capsule architecture to better learn the relationships between object parts and the whole, and make the model more robust and better generalizable.

Recent findings in neuroscience show that, monkey inferotemporal (IT) cortex (the final stage of the “what" pathway of human and monkey visual system) contains separate regions where neurons exhibit disentangled representations of shape and appearance properties of face images, respectively. This disentangled representation has been applied to construct generative models of images  <cit.>, in which sampled disentangled embedding vectors are passed to a generator (decoder) to generate images. 

In this paper, we take the above neuroscience inspiration to regularize the embedding layer of a traditional CNN classifier by a disentangled deep autoencoder for image classification. 
In the regularizing deep autoencoder, an encoder disentangles an input image into a low-dimensional appearance code capturing the texture or background information and a low-dimensional geometric code capturing the category-sensitive shape information, and two decoders reconstruct the original high-dimensional input image from the disentangled low-dimensional code. 

Our proposed regularization framework improves CNNs' robustness and generalization mainly due to the following two reasons: First, due to the large representational capability of traditional CNNs, there are a lot of embedding functions including many pathological ones from high-dimensional input image space to the low-dimensional embedding space that can equally well minimize the classification loss; the reconstruction loss of the deep autoencoder imposes strong regularization on the low-dimensional embedding code directly used for classification; Second, the disentanglement in the embedding space imposes additional regularization on the low-dimensional code, resulting in specialized clean appearance code and geometric code, respectively, which might be, respectively, more discriminative for different classification tasks than a low-dimensional code containing a lot of information mixed with noise.

Our proposed regularization framework significantly improves the robusness of CNNs against both white-box attacks <cit.>, which assume the model is completely known, and black-box attacks <cit.>, which  can only query the model for a limited number of times.

Our contributions in this paper are as follows: (i) We propose the first disentangling based deep autoencoding regularization framework to improve the robustness of CNNs, which can be easily applied to any embedding based CNN classification model and combined with standard defense methods; (ii) Extensive experiments show that, our regularization framework dramatically improves the robustness of standard CNNs against the attacks from adversarial examples; (iii) Our proposed framework helps CNNs learn more robust image representation and gain better generalization to novel test data with large geometric differences, achieving state-of-the-art generalization ability compared with capsule networks <cit.>.









§ METHOD

In this section, the proposed framework is described in detail.


 §.§ Inspiration from neuroscience perspective

Recently, Chang and Tsao's work <cit.> showed that primate IT cortex forms patches of neurons that separately encode appearance features (roughly speaking, corresponding to textures and their coarse spatial organization) and shape features (spatial warping) for face images. Importantly, the representations of appearance and shape are disentangled, e.g., neurons coding for appearance are invariant to geometric variations of images such as stretching and rotating. Such disentangling property may have ecological relevance, because appearance and shape contain different types of information: stable information such as skin color and texture are more reflected in the appearance of images, whereas other types of information such as pose and rotation and sometimes facial expression are more reflected in the shape (geometric) properties.
stable information such as object identity and category are more reflected in the appearance of images, whereas other types of information that change more frequently such as facial expression and pose of objects are more reflected in the shape (geometric) properties. 
It has been found that when understanding movies and stories, the brain uses a hierarchical representation for changes in environment at different time scales  <cit.>. Encoding image information relevant for tasks at different time scales separately may be necessary for an agent to effectively allocate its computational resources. For example, for a driver to decide whether a pedestrian is jumping into the street, he/she only needs to pay attention to the change of the pose of the pedestrian (which is mostly characterized in the shape of body) instead of the identity of the pedestrian (which is mostly reflected in the appearance). If a fixed total size of representational space is necessary to encode both shape and appearance, disentangling them in separate subspaces reduces the dimensionality of the space that a downstream neural network needs to learn a mapping to the properties critical for its task. Reduced relevant dimension means fewer training samples needed in order to achieve the same level of generalization. The brain likely has feedback information from natural tasks to encourage disentangled representation, while such feedback does not exist when training CNN to classify static images. Therefore, we propose to introduce augmentation tasks in Section <ref> that encourage the representation learned by the CNN to exhibit similar disentangling property as that of primate IT cortex.


 §.§ Disentangled Deep Autoencoding Regularization



Suppose we have a K-class training set D_T = {(𝐈^i, 𝐲^i), i=1,⋯,N} containing N pairs of samples, where 𝐈^i ∈ℝ^H × W × 3 is the image to be classified and 𝐲^𝐢∈{0,⋯,K-1} is the corresponding label. Given a new test image 𝐈^𝐣 that does not belongs to D_T, the classification task is to predict the correct label 𝐲^𝐣 based on the features 𝐂^𝐣 extracted from 𝐈^𝐣.

To encourage learning visual features robust to natural variations of images, we employ additional learning objective and augmentation task to force a classifier to learn disentangled geometric code and appearance code.
As shown in Figure <ref>, the proposed framework contains two parts. The first part is a regularizing autoencoder with disentangled feature embedding, displayed on the top. The second part performs classification on the concatenated embedding feature vector, displayed on the bottom.
For any image 𝐈^𝐢, the encoder maps it to a pair of appearance and geometric codes as follows,

    𝐂_a, 𝐂_g = Encoder(𝐈),

in which our encoder learns a disentangled embedding of the input image, namely the appearance code 𝐂_a and the geometric code 𝐂_g. The appearance decoder or generator takes the appearance code to generate the appearance output 𝐗_a. The geometric decoder takes the geometric code and generates coordinate displacement of the appearance output 𝐗_g.

    𝐗_a = Decoder_A(𝐂_a),
        𝐗_g = Decoder_G(𝐂_g).

The warping function 𝐅_W warps the appearance with the coordinate displacement generated by the geometric decoder and generates the reconstruction output 𝐗. This will be described in detail later.

    𝐗 = 𝐅_W(𝐗_a,𝐗_g).

The classifier 𝐅_CLS takes both the appearance code and geometric code as input and predicts category probability 𝐲̂.

    𝐲̂ = 𝐅_CLS(𝐂_a, 𝐂_g).


In training process, the encoder, two decoders as well as classifier are trained jointly and the loss function ℒ is defined as

    ℒ = ℒ_crossentropy(𝐲̂, 𝐲) + λ‖𝐗 - 𝐈‖,

where 𝐲 is the training label vector, ‖𝐗 - 𝐈‖ is the reconstruction error (frobenius norm), and ℒ_crossentropy is the cross-entropy loss, and λ is the regularization weight for the reconstruction loss.

The disentangled representation is learned in an unsupervised manner. The warping function naturally encourages the separation of appearance code from geometric code because 𝐗_g represents coordinate displacement of 𝐗_a.

Note that the complete architecture is needed only for training. For inference, the two decoders, warping function and the reconstruction are not needed.



  §.§.§ Warping Function


The warping function, such as the one used in Spatial Transformer networks (STN) <cit.>, is usually a combination of geometric transformation for pixels and interpolating operation.
The geometric transformation usually consists of two  types: affine transformation and pixel displacement transformation (like the optical flow field). We employ the second type in this work, similarly as in  <cit.>. 
The warping function takes the displacement field and the "unwarped" image as the input. The vector at each pixel of the displacement field is defined to be related with the spatial shift needed to move the destination pixel (x,y) to source pixel (x_0,y_0) in the output of the appearance generator as below:

    Φ_𝐠(
        x
    y
        )=
        (
        x+dx
    y+dy
        )=
        (
        x_0
    y_0
        ).

where Φ_𝐠 denotes the pixel displacement operation.

The outputs of the pixel displacement operation are usually decimals. In order to get a set of pixels whose coordinates are integers, we employ the differentiable bi-linear interpolation in this work. The bi-linear interpolation process can be represented as:

    𝐗(x,y) =    Σ_h=1^HΣ_w=1^W 𝐗_a(h,w) M(1-| x_0 -w|)
        M(1-| y_0-h|).

where M(·) means the function max(0,·), and H, W are the height and width of the input image respectively.

In this way, various pictures of the same object can be encoded with similar appearance code, and their differences in viewing angles, locations, and deformation are captured by the geometric code. With this idea, we use data augmentation detailed below to enforce the separation of appearance and geometric information.  



 §.§ Data Augmentation and Code Regularization


In this section, we will describe how to combine the proposed regularization framework with deformation-based data augmentation method.
This deformation-based data augmentation is achieved by random 2D-stretching, scaling and spatial shift of the target images. To encourage appearance code to capture information invariant to spatial transformation, we impose L2 regularization on the distance between the appearance codes of images before (denoted as 𝐈) and after deformation (𝐈').

The appearance code and geometric code of 𝐈' from encoder are 𝐂'_a and 𝐂'_g. And the corresponding reconstruction output and classification output are denoted as 𝐗' and 𝐲̂'. The new loss function ℒ_AUG is 

    ℒ_AUG = ℒ   +  ℒ_crossentropy(𝐲̂', 𝐲)+ λ‖𝐗' - 𝐈'‖
       +γ‖𝐂_a - 𝐂'_a‖.

where γ is the scale weight for appearance code regularization. We set λ and γ as 1 in experiments.

As shown in Section <ref>, data augmentation and code regularization further encourages the separation of appearance code from geometric code. This leads to much improved generalization to novel test data with large geometric transformation.



§ EXPERIMENTS


In this section, the robustness and generalization to novel test data of the proposed regularization framework is evaluated. In addition, ablation studies are performed to analyze the individual mechanisms that contribute to the gains. 



 §.§ Robustness Against Adversary Attack

We evaluate our framework's robustness against both white-box and black-box attacks. White-box attacks assume that the adversary knows the network architecture and the model parameters after training or has access to labeled training data. An adversary can use the detailed information to construct a perturbation for a given image. Black-box attacks only rely on a limited number of queries to the model to construct an adversarial image.

To compare with the related work such as  <cit.>, we use three dataseets, the MNIST dataset, the FaceScrub dataset, and the CIFAR10 datasset. The MNIST dataset of handwriten digits has 60000 for training and 10000 for test. The FaceScrub data set contains 106,863 face images of 530 celebrities taken under real-world conditions. The CIFAR10 dataset contains 60,000 color images with a resolution of 32x32 in 10 different classes, in which 50,000 are used for training and 10,000 are used for test.



  §.§.§ White-Box Attack

We evaluate our model's robustness using popular white box attack methods: fast gradient sign method (FGSM) <cit.> and Basic Iterative Method (BIM) <cit.>.  FGSM computes the gradient of the loss with respect to each pixel intensity and then changes the pixel intensity by a fixed amount ϵ in the direction that increases the loss.  BIM applies FGSM with multiple smaller steps when creating the adversarial image. FGSM and BIM are adopted as the main attack methods by a series of works. Since these methods attack the image based on the gradient, they pose a great threat to modern CNN architecture trained through gradient back-propagation.

We use the same benchmark CNN architecture as  <cit.>. It has 4 convolutional layers with a softmax output. For fair comparison, the classifier part of the proposed framework uses exactly the same architecture as the benchmark CNN architecture. To test the robustness of the proposed framework, adversarial images are generated from the test set using a fully trained model. We then reported the accuracy of the model on these images. All models are trained without adversary examples. 

As show in Table <ref> and  <ref>, the proposed regularization framework is much more robust than the benchmark CNN architecture. [The visualization of feature embedding of legitimate images and adversarial images on the MNIST dataset can be found in the supplementary material.]




Clearly we can further improve the performance of the proposed model by adversary training. However, this is out of the scope as our main goal is not attack defense. Our main objective is to design discriminative models that learn representations that are robust and easier to generalize to novel test data.


It's worth noting that for defending against the adversary attack with large perturbation, simply adding the proposed regularization even exceeds some defense models that used adversary training. For example, our model shows a higher performance (61.5%) over the gradient-regularized model <cit.> (lower than 60%) when perturbed with a big perturbation (0.5) by FGSM on MNIST Dataset.

We further evaluate our model's robustness using FaceScrub dataset. To simplify the problem, we randomly select 50 faces for the classification task. For the benchmark CNN architecture, the training and test accuracy is 99.9% and 74.2%, respectively. While after adding the proposed regularization, the training and test accuracy is 99.9% and 79.8%, respectively. As shown in Table <ref> and  <ref>, the regularization framework is much more robust than the benchmark CNN model.
Please note that ϵ = 0.05 is big enough as a perturbation in this task.




Finally, the framework is tested with CIFAR10 dataset to evaluate the proposed disentangling method on general natural images where the information of categories can lie in both appearance code and geometric code in varying degrees. In this experiment, data augmentation methods including random cropping and random flipping are applied to the normalized input data. We use 110-layer ResNet <cit.> as the baseline and it achieves 8.5% error rate on the test set. To add the proposed regularization, the output of the final fully-connected layer of the ResNet is modified to generate geometric and appearance code. The decoder is the same as described in the main text. With our proposed special regularization, the network achieves 8.2% error rate. The defense result is shown as below:




As shown in Table <ref> and Table <ref>, our method outperforms the CNN baseline for the FGSM attack and BIM attack. The proposed method improves over CNN baseline by 15% to 37% for FGSM attack.



  §.§.§ Black-Box Attack

The first black-box attack method we use is proposed in  <cit.>. The basic idea is greedy local search. It is an iterative search procedure, where in each round a local neighborhood is used to refine the current image and to optimize some objective function depending on the network output. The number of iterations of the black-box attack is adjusted as an experimental variable. As shown in Table <ref>, the proposed regularization method can significantly promote the benchmark CNN architecture.



Besides the greedy local search black-box attack approach, another black-box attack approach <cit.> leverages transferability property in creating adversarial examples. The attack first creates attack examples using a known deep learning model (white-box attack). The attack then apply these adversarial examples on the deep learning model with no access to model architecture and parameters. We tried this type of black-box attack and observed that our method has no noticeable advantage over benchmark CNN architecture. This is not surprising as transferability property applies to both deep learning models and non-deep learning models such as decision trees and SVM in general <cit.>.




  §.§.§ Joint Defense


As has been stated, the proposed framework is orthogonal to other defense techniques. It can potentially be combined with these defense techniques to further improve the robustness of CNN models. As an example, we combine it with adversarial training. Specifically, we train the model with legitimate inputs (MNIST) and adversarial samples generated by the model using FGSM attack method with ϵ of 0.3 <cit.>. 


As shown in Table <ref>, the result proves that our regularization method could be combined with adversarial training to further improve the robustness of the classifiers. Our method with no adversarial training (see Table 1 in the main paper) outperforms CNN baseline with adversarial training (See Table <ref>). Our method with adversarial training (See Table <ref>) further outperforms our method without adversarial training (see Table 1 in the main paper). For example, it is 34.8% vs 38.5% for ϵ = 0.5.




  §.§.§ Robust feature embedding against attacks

To verify that our model indeed has a more robust feature embedding, we used t-SNE  <cit.> to visualize the embedding of attacked images against unattacked images from MNIST dataset for both baseline model and our model in Figure  <ref>. Indeed, the embedding in the joint appearance and geometric code in our model for attacked images (<ref>d) maintains stronger separability compared to baseline CNN (<ref>b). 
One of the reasons that we expect our model to be more robust against adversarial attack is that appearance and geometric features may each be more suitable for different classification tasks. Take digits as examples, several digits share the same topological structure but mainly differ in their geometric properties. By separating the feature space to 𝐂_a and 𝐂_g, we provide to the classifier 𝐅_CLS  shape codes 𝐂_g which live on a lower dimensional space than needed for traditional CNN, which makes the training more data-efficient. Interestingly, this is also reflected in the embedding structures: as shown in Figures <ref>e and <ref>g, even before attack, manifolds of different digits are entangled in the space of 𝐂_a. But as shown in 3f and 3h, the embedding in 𝐂_g stays highly separable even after attack, suggesting the classifier indeed is able to utilize the more robust code 𝐂_g.




 §.§ Generalization


We evaluate our model's generalization to novel test data with large geometric transformation on the smallNORB dataset<cit.>. The dataset contains five categories of 3D objects (animals, human, airplanes, trucks and cars), each with 10 instances. These images are captured through a combination of six illumination conditions, nine elevations, and eighteen azimuths and have a resolution of 96 × 96. The training set and the testing set contain 5 different instances of each category. This data set requires the model to learn the common features within each target category.



For ease of comparison, we use the same experimental setting and the same network architecture of the benchmark CNN as reported in <cit.>. Both our model and baseline CNN model are trained on one-third of the training set, which contains six smallest azimuths or 3 smallest elevations, and tested on two-thirds of the test set, which contains images of totally different azimuths and elevations from those in training set. For training, we randomly crop 48 × 48 sub-patch from the input image, resize the patch into 32 × 32 and add small random jitters to brightness and contrast. For testing, the samples are directly cropped from the center. For both training and testing, all data have been normalized to have zero mean and unit variance.

To better disentangle geometric code from appearance code, we use the data augmentation and code regularization technique proposed in Section <ref>. Specifically, deformations such as displacement and stretching to the inputs is applied. The extent of stretching in this experiment is randomly selected from {2, 4, 6, 8} pixels, the distance of displacement in each dimension (in the range of {2, 4, 6, 8}) and the orientation of displacement are also randomly selected. Then the samples are re-sized back to 32 × 32 and padded by pixels at the edge when necessary. These deformation operations do not change the appearance contents. The L2 norm of the difference between appearance codes of an input image and those corresponding to its deformed image are used as a regularization term as in Section <ref>.  


As shown in Table <ref>, our model is 8% better in accuracy when generalizing to novel elevation and 2% better in case of novel azimuth. We note that of particular challenge, the task is to generalize to novel elevation and azimuth. The test distribution is different from the training distribution which makes the classification extremely hard. 

For example, Figure <ref> shows the cars in testing set with the highest visual similarity to example cars in training set. With largely different azimuths and elevations and even shapes, the generalization task can be difficult even for human observers.



Note that our CNN baseline performance on novel elevation is lower than that reported in  <cit.> while the performance on novel azimuth is higher. Although we use exactly the same network architecture according to the description in  <cit.>, such difference may be unavoidable given that many parameters such as brightness and contrast jittering range need to be tuned, which were not mentioned in the paper. 
Our result on generalizing to novel azimuth achieves state-of-the-art. We achieve 86.6%, comparable to 86.5%  of CapsuleNet as reported in <cit.>. For generalizing to novel elevation, even compared with the CNN benchmark result of 82.2% reported in  <cit.>, our result of 85.3% is still 3.7% better. It is worth noting that our method is orthogonal to the capsule technique  <cit.>. Potentially the strength of feature disentangling and the capsule design may be combined to further improve performance.



 §.§ Synthesis Study

To evaluate the quality of disentangling, we examine the property of "unwarped" face image 𝐗_a and the exchangeability of 𝐂_g and 𝐂_a.

Two faces (𝐈^0 and 𝐈^1) are randomly selected and encoded by our model as 𝐂_a^0, 𝐂_g^0) and (𝐂_a^1, 𝐂_g^1). We then exchange their codes into (𝐂_a^0, 𝐂_g^1) and (𝐂_a^1, 𝐂_g^0). The swapped codes are then decoded by the appearance and geometric decoder and new images are synthesized by the warping function: 𝐗^i_new = 𝐅_W(𝐃𝐞𝐜𝐨𝐝𝐞𝐫_𝐀(𝐂_a^i), 𝐃𝐞𝐜𝐨𝐝𝐞𝐫_𝐆(𝐂_g^1-i)), where i ∈{0, 1}.











Two pairs of (𝐗^0_new,  𝐗^1_new) are visualized on Figure <ref>. The appearance output is 𝐃𝐞𝐜𝐨𝐝𝐞𝐫_𝐀(𝐂_a^i), which is the "unwarped" image. The reconstruction output is the warped output without swapping code (𝐗^i) and the swap code output is the the warped output after swapping code (𝐗^i_new). Group 1 and Group 2 emphasize the effects of maintaining geometric code and appearance code, respectively, while exchanging the other code.
This figure prominently reflects the role of the geometric code and the appearance code in the face reconstruction process of the proposed model. It can be observed that the geometric code mainly contains some deformation information, such as the facial features, the expression and even the face orientation. On the other hand, the appearance code contains texture information, such as skin color, hair color, background as well as the basic structure of face.



 §.§ Ablation Studies


We conduct ablation studies to analyze the relative contributions of each design component in our framework: disentangled representation learning, augmented training, L2 regularization of appearance code difference when an input image is deformed. 
We first investigate the benefit of disentangled feature learning in terms of robustness to FSGM attack using MNIST dataset. To disable disentangled feature learning, the warping function component is removed from Figure <ref>. We refer it as 'Reconstruction' in Table  <ref>.
As shown in Table <ref>, disentangled feature learning is essential for the model to be robust against FGSM, e.g. 16.15% success rate vs 43.83% success rate at ϵ=0.3.





Another ablation study are further conducted on generalization using smallNORB dataset.  The experimental settings is the same as section <ref>. The base model in Table <ref> is the architecture in Figure <ref> with no augmented training and additional regularization. The augmented training settings in Table  <ref> refers to training with augmented data by deforming input data. The augmented training and code regularization setting in Table <ref> means that the L2 regularization of appearance code difference between an input image and its deformed one is further imposed.

Results in Table <ref> show that with the proposed regularization framework, the CNN base model is promoted in all settings. It also shows that augmented training and additional regularization helps. With both augmented training and L2 regularization, the improvement over the base model is 2.2%.



§ RELATED WORK

Theoretical work on robustness and generalization: Recently In  <cit.>, a method is proposed to learn deep ReLU-based classifiers that are provably robust against norm-bounded adversarial perturbations on the training data. However, the method still can not scale to realistic network sizes.

 Neuroscience inspired architectures:  Recently Chang and Tsao's work <cit.> has inpired <cit.>. Xing et al. <cit.> proposed a deformable generator model to disentangle the appearance and geometric information from images into two independent latent vectors. They showed that both representations can be learned in an unsupervised manner.  Han et al.  <cit.> showed that a generator network can learn the appearance and shape model in  <cit.>. Our work is an extension in this direction by directly enforcing the disentangling on the features used by a discriminative network for image classification. Hinton <cit.> leveraged capsule, a group of neurons to better learn the parts-whole relationship of objects. This makes the model more robust and easier to generalize. 

Emperical defense methods: 
Athalye et al.  <cit.> identify obfuscated gradients, a phenomenon exhibited by certain defenses that makes standard gradient-based methods fail
to generate adversarial examples. They develop three attack techniques to bypass three different types of obfuscated gradients, circumventing defenses <cit.>. Many defenses also perform adversarial training. Our method is orthogonal to these defense methods. 

Learning disentangled feature representation:
There has been several works on learning disentangled feature representation in the machine learning literature.  <cit.> and <cit.> disentangled video representation into content and pose. 

<cit.> disentangled video representation into content and motion using image differences for video prediction. 
<cit.> learn disentangled representations using analogy reasoning. Unlike previous methods that model pose with a rigid affine mapping, the coordinate displacement in our work is a non-rigid deformable mapping. In addition, most of previous methods require learning from video data in order to utilize the time-dependent features in pose and time-independent ones in content. Our model does not require time-series data and is suitable for independently sampled image data. Finally, our proposed framework aims to improve the robustness of CNNs, which has a completely different purpose from previous models.

Adversarial attack methods: 
Szegedy et al. <cit.> proposed and defined the adversarial attack problem as to find the small perturbation to real images that makes the well-trained neural network misclassifying perturbed images. Current adversarial attacks are broadly classified into white-box attacks and black-box attacks. White-box attacks assume the model is completely known. Black-box attacks can only query the model for a limited number of times. The most widely used methods for evaluating model robustness is the fast gradient sign method (FGSM) <cit.> and the Basic Iterative Method (BIM) <cit.>. Jacobian-based Saliency Map attack (JSMA) <cit.> computes a saliency map as a response to per-pixel modification and achieve targeted white box attack by altering the pixels in saliency map. To attack the distillation defense method <cit.>, Carlini and Wagner attacks (CW)  <cit.> restrict l_2, l_∞, and  l_0 norms to make the perturbation quasi-imperceptible.



For ease of comparison with related work, we choose FGSM and BIM.
Black-box attacks are based on a limited knowledge of the target model. The greedy local search method <cit.> uses an iterative search procedure, where in each round a local neighborhood is used to refine the current image and to optimize some objective function depending on the network output. The second black-box attack approach <cit.> leverages transferability property where a different known model is used to generate adversarial examples. We evaluate our model robustness using both types of black-box attacks.


§ CONCLUSION AND FUTURE WORK

Robustness and generalization are paramount to deploying deep learning models in mission-critical applications. 
In this paper, we take inspiration from neuroscience and show that disentangling feature representation into appearance and geometric code can significantly improve the robustness and generalization of CNNs.

There are other mechanisms at play for the robustness and generalization ability of primate brain's perception. In the future, we plan to explore other neuroscience motivated directions. For example, our model can be potentially combined with capsule networks <cit.>.


named





§ SUPPLEMENTARY MATERIAL: ROBUST FEATURE EMBEDDING AGAINST ATTACKS

To verify that our model indeed has a more robust feature embedding, we used t-SNE  <cit.> to visualize the embedding of adversarial images against legitimate images from MNIST dataset for both baseline model and our model in Figure  <ref>. Indeed, the embedding in the joint appearance and geometric code in our model for attacked images (<ref>d) maintains stronger separability compared to baseline CNN (<ref>b). 
One of the reasons that we expect our model to be more robust against adversarial attack is that appearance and geometric features may each be more suitable for different classification tasks. Take digits as examples, several digits share the same topological structure but mainly differ in their geometric properties. By separating the feature space to 𝐂_a and 𝐂_g, we provide to the classifier 𝐅_CLS  shape codes 𝐂_g which live on a lower dimensional space than needed for traditional CNN, which makes the training more data-efficient. Interestingly, this is also reflected in the embedding structures: as shown in Figures <ref>e and <ref>g, even before attack, manifolds of different digits are entangled in the space of 𝐂_a. But as shown in 3f and 3h, the embedding in 𝐂_g stays highly separable even after attack, suggesting the classifier indeed is able to utilize the more robust code 𝐂_g.



