Question:
represent the text to find a very short summary (1-2 sentences max).
The query could be 'Self-supervised learning (SlfSL), aiming at learning feature representations through ingeniously designed pretext tasks without human annotation, has achieved compelling progress in the past few years. Very recently, SlfSL has also been identified as a promising solution for semi-supervised learning (SemSL) since it offers a new paradigm to utilize unlabeled data. This work further explores this direction by proposing a new framework to seamlessly couple SlfSL with SemSL. Our insight is that the prediction target in SemSL can be modeled as the latent factor in the predictor for the SlfSL target. Marginalizing over the latent factor naturally derives a new formulation which marries the prediction targets of these two learning processes. By implementing this framework through a simple-but-effective SlfSL approach -- rotation angle prediction, we create a new SemSL approach called Conditional Rotation Angle Prediction (CRAP). Specifically, CRAP is featured by adopting a module which predicts the image rotation angle \textbf{conditioned on the candidate image class}. Through experimental evaluation, we show that CRAP achieves superior performance over the other existing ways of combining SlfSL and SemSL. Moreover, the proposed SemSL framework is highly extendable. By augmenting CRAP with a simple SemSL technique and a modification of the rotation angle prediction task, our method has already achieved the state-of-the-art SemSL performance. The recent success of deep learning is largely attributed to the availability of a large amount of labeled data. However, acquiring high-quality labels can be very expensive and time-consuming. Thus methods that can leverage easily accessible unlabeled data become extremely attractive. Semisupervised learning (SemSL) and self-supervised learning (SlfSL) are two learning paradigms that can effectively utilize massive unlabeled data to bring improvement to predictive models. SemSL assumes that a small portion of training data is provided with annotations and the research question is how to use the unlabeled training data to generate additional supervision signals for building a better predictive model. In the past few years, various SemSL approaches have been developed in the context of deep learning. The current state-of-the-art methods, e.g. MixMatch , unsupervised data augmentation , converge to the strategy of combining multiple SemSL techniques, e.g. Π-Model , Mean Teacher , mixup , which have been proved successful in the past literature. SlfSL aims for a more ambitious goal of learning representation without any human annotation. The key assumption in SlfSL is that a properly designed pretext predictive task which can be effortlessly derived from data itself can provide sufficient supervision to train a good feature representation. In the standard setting, the feature learning process is unaware of the downstream tasks, and it is expected that the learned feature can benefit various recognition tasks. SlfSL also offers a new possibility for SemSL since it suggests a new paradigm of using unlabeled data, i.e., use them for feature training. Recent work has shown great potential in this direction. This work further advances this direction by proposing a new framework to seamlessly couple SlfSL with SemSL. The key idea is that the prediction target in SemSL can serve as a latent factor in the course of predicting the pretext target in a SlfSL approach. The connection between the predictive targets of those two learning processes can be established through marginalization over the latent factor, which also implies a new framework of SemSL. The key component in this framework is a module that predicts the pretext target conditioned on the target of SemSL. In this preliminary work, we implement this module by extending the rotation angle prediction method, a recently proposed SlfSL approach for image recognition. Specifically, we make its prediction conditioned on each candidate image class, and we call our method Conditional Rotation Angle Prediction (CRAP). The proposed framework is also highly extendable. It is compatible with many SemSL and SlfSL approaches. To demonstrate this, we further extend CRAP by using a simple SemSL technique and a modification to the rotation prediction task. Through experimental evaluation, we show that the proposed CRAP achieves significantly better performance than the other SlfSL-based SemSL approaches, and the extended CRAP is on par with the state-of-the-art SemSL methods. In summary, the main contributions of this paper are as follows: • We propose a new SemSL framework which seamlessly couples SlfSL and SemSL. It points out a principal way of upgrading a SlfSL method to a SemSL approach. • Implementing this idea with a SlfSL approach, we create a new SemSL approach (CRAP) that can achieve superior performance than other SlfSL-based SemSL methods. • We further extend CRAP with a SemSL technique and an improvement over the SlfSL task. The ed new method achieves the state-of-the-art performance of SemSL. Our work CRAP is closely related to both SemSL and SlfSL. SemSL is a long-standing research topic which aims to learn a predictor from a few labeled examples along with abundant of unlabeled ones. SemSL based on different principals are developed in the past decades, e.g., "transductive" models , multi-view style approaches and generative model-based methods , etc. Recently, the consistency regularization based methods have become quite influential due to their promising performance in the context of deep learning. Specifically, Π-Model requires model's predictions to be invariant when various perturbations are added to the input data. Mean Teacher enforces a student model producing similar output as a teacher model whose weights are calculated through the moving average over the weight of student model. Virtual Adversarial Training encourages the predictions for input data and its adversarially perturbed version to be consistent. More recently, mixup has emerged as a powerful SemSL regularization method which requires the output of mixed data to be close to the output mixing of original images. In order to achieve good performance, most state-of-the-art approaches adopt the strategy of combining several existing techniques together. For example, Interpolation Consistency Training incorporates Mean Teacher into the mixup regularization, MixMatch adopts a technique that uses fused predictions as pseudo prediction target as well as the mixup regularization. Unsupervised data augmentation upgrades Π-Model with advanced data augmentation methods. SlfSL is another powerful paradigm which learns feature representations through training on pretext tasks whose labels are not human annotated. Various pretext tasks are designed in different approaches. For example, image inpainting trains model to reproduce an arbitrary masked region of the input image. Image colorization encourages model to perform colorization of an input grayscale image. Rotation angle prediction forces model to recognize the angle of a rotated input image. After training with the pretext task defined in a SlfSL method, the network is used as a pretrained model and can be fine-tuned for a downstream task on task-specific data. Generally speaking, it is still challenging for SlfSL method to achieve competitive performance to fully-supervised approaches. However, SlfSL provides many new insights into the use of unlabeled data and may have a profound impact to other learning paradigms, such as semi-supervised learning. SlfSL based SemSL is an emerging approach which incorporates SlfSL into SemSL. The most straightforward approach is to first perform SlfSL on all available data and then fine-tune the learned model on labeled samples. S 4 L ) is a newly proposed method which jointly train the downstream task and pretext task in a multi-task fashion without breaking them into stages. In this paper, we further advance this direction through proposing a novel architecture which explicitly links these two tasks together and ensure that solving one task is beneficial to the other. In SemSL, we are given a set of training samples {x 1, x 2, · · ·, x n} ∈ X with only a few of them X l = {x 1, x 2, · · ·, x l} ∈ X annotated with labels {y 1, y 2, · · ·, y l} ∈ Y l (usually l << n and y is considered as discrete class label here). The goal of a SemSL algorithm is to learn a better posterior probability estimator over y, i.e., p(y|x, θ) with θ denoting model parameters, from both labeled and unlabeled training samples. SlfSL aims to learn feature representations via a pretext task. The task usually defines a target z, which can be derived from the training data itself, e.g., rotation angle of the input image. Once z is defined, SlfSL is equivalent to training a predictor to model p(z|x; θ). There are two existing schemes to leverage SlfSL for SemSL. The first is to use SlfSL to learn the feature from the whole training set and then fine-tuning the network on the labeled part. The other is jointly optimizing the tasks of predicting y and z, as in the recently proposed S 4 L method. As shown in Figure 1 (a), S 4 L constructs a network with two branches and a shared feature extractor. One branch for modeling p(y|x; θ) and another branch for modeling p(z|x; θ). However, in both methods the pretext target z predictor p(z|x; θ) is implicitly related to the task of predicting y. Our framework is different in that we explicitly incorporate y into the predictor for z. Specifically, we treat y as the latent factor in p(z|x; θ) and factorize p(z|x; θ) through marginalization: Eq. 1 suggests that the pretext target predictor p(z|x; θ) can be implemented as two parts: a model to estimate p(y|x; θ) and a model to estimate z conditioned on both x and y, i.e., p(z|x, y; θ). For the labeled samples, the ground-truth y is observed and can be used for training p(y|x; θ). For unlabeled samples, the estimation from p(y|x; θ) and p(z|x, y; θ) will be combined together to make the final prediction about z. Consequently, optimizing the loss for p(z|x; θ) will also provide gradient to back-propagate through p(y|x; θ). This is in contrast to the case of S 4 L, where the gradient generated from the unlabeled data will not flow through p(y|x; θ). Theoretically, p(z|x; θ) and p(y|x; θ) can be two networks, but in practise we model them as two branches connecting to a shared feature extractor. p(z|x; θ) suggested by Eq. 1 is essentially a pretext target predictor with a special structure and partial observations on its latent variable, i.e. y. The benefits of using such a predictor can be understood from three perspectives: p(y|x; θ) in Eq. 1 acts as a soft selector to select p(z|x, y; θ) for predicting z. If the estimation of p(y|x; θ) is accurate, it will select p(z|x, y =ŷ(x); θ) for prediction and update, whereŷ(x) is the true class of x. This selective updating will make p(z|x, y; θ) give more accurate prediction over z if y matchesŷ(x). After such an update, p(z|x, y; θ) will in turn encourage p(y|x; θ) to attain higher value for y =ŷ(x) since the prediction from p(z|x, y =ŷ(x); θ) is more likely to be accurate. Thus, the terms p(y|x; θ) and p(z|x, y; θ) will reinforce each other during training. even if p(y|x; θ) is not accurate (this may happen at the beginning of the training process), p(z|x, y; θ) can still perform the pretext target prediction and act as an unsupervised feature learner. Thus, the features will be gradually improved in the course of training. With a better feature representation, the estimation of p(y|x; θ) will also be improved. Finally, to predict z in Eq. 1, p(z|x, y; θ) needs to be evaluated for each candidate y. This in effect is similar to creating an ensemble of diversified pretext target predictors and with the combination weight given by p(y|x; θ) according to the marginalization rule. Thus, training features with Eq. 1 may enjoy the benefit from ensemble learning. Again, this will lead to better features and thus benefit the modelling of p(y|x; θ) and p(z|x, y; θ). The above framework provides a guideline for turning a SlfSL method into a SemSL algorithm: modifying a SlfSL predictor p(z|x; θ) by p(z|x, y; θ) and introducing a branch for p(y|x; θ) optimizing the prediction of z on the SemSL dataset and update the branches p(z|x, y; θ), p(y|x; θ) and their shared feature extractor. using p(y|x; θ) as downstream task predictor or adding an additional branch for training p(y|x; θ) only with the labeled data as in S 4 L. More details about the additional branch will be explained in Section 4. < l a t e x i t s h a 1 _ b a s e 6 4 = " t 2 W 6 x u / Y c G q z 7 e 6 w r U f 4 A 7 6 z e 2 A = " < l a t e x i t s h a 1 _ b a s e 6 4 = " t 2 W 6 x u / Y c G q z 7 e 6 w r U f 4 A 7 6 z e 2 A = " In the following part, we will describe an implementation of this framework, which is realized by upgrading the rotation-angle prediction-based SlfSL to its conditional version. Rotation angle prediction is a recently proposed SlfSL approach for image recognition. It randomly rotates the input image by one of the four possible rotation angles ({0 •, 90 •}) and requires the network to give a correct prediction of the rotation angle. Despite being extremely simple, this method works surprisingly well in practice. The underlying logic is that to correctly predict the rotation angle, the network needs to recognize the canonical view of objects from each class and thus enforces the network to learn informative patterns of each image category. Following the proposed framework, we upgrade rotation angle prediction to conditional rotation angle prediction (CRAP) for semi-supervised learning. In this case, z in Eq. 1 is the rotation angle and y is the class label of input image x. We realize p(z|x, y; θ) by allocating a rotation angle prediction branch for each class. The prediction from each branch is then aggregated with the aid of p(y|x; θ) for the final prediction of z as shown in Eq. 1. A more detailed schematic illustration of the CRAP method is shown in Figure 1 (b). As seen, our method adopts a network with multiple branches and a shared feature extractor. Specifically, branches within the dashed box are called auxiliary branches since they are only used for training and will be discarded at the test stage. It contains C rotation predictors which corresponds to p(z|x, y; θ) and a semantic classifier which generates p(y|x; θ). The auxiliary branches and feature extractor are trained by using the procedure described in Section 3. Note that in CRAP, we do not directly use the semantic classifier from the auxiliary branches as the final classifier. Instead, we introduce an additional semantic classifier and learn it only via the loss incurred from the labeled data. This treatment is similar to S 4 L and we find this strategy work slightly better in practice. We postulate the reason is that the p(y|x; θ) branch in auxiliary branches is mainly trained by the supervision generated from the optimization of p(z|x; θ). Such supervision is noisy comparing with the loss generated from the ground-truth y. It is better to use such a branch just for feature training since the latter is more tolerant to noisy supervision. Remark: One potential obstacle of our model is that the quantity of parameters in the auxiliary branches would increase significantly with a large C. To tackle this, we propose to perform dimension reduction for the features feeding into the rotation predictor. Results in Section 5.3 show that this scheme is effective as our performance will not drop even when the dimension is reduced from 2048 to 16. The CRAP method is also highly expendable. In the following, we will extend CRAP from two perspectives: improving p(y|x; θ) and improving p(z|x, y; θ). As discussed in Section 3, our method essentially introduces a network module with a special structure and partial observations on the latent variable y. Besides using labeled data to provide supervision for y, we can also use existing SemSL techniques to provide extra loss for modeling p(y|x; θ). To implement such an extension, we employ a simple SemSL loss as follows: we rotate each image in four angles within one batch (the prediction of the rotated image can be obtained as the byproduct of CRAP) and obtain the arithmetic averagep of the predicted distributions across these four rotated samples. Then we perform a sharpening operation overp as in MixMatch: where C is the number of classes and T ∈ is a temperature hyper-parameter. Then we use the cross entropy betweenp i and p(y|x; θ) (in auxiliary branches) as an additional loss. Note that other (more powerful) SemSL can also apply here. We choose the above SemSL technique is simply because its operation, i.e. image rotation, has already been employed in the CRAP algorithm and thus could be reused to generate the additional SemSL loss without increasing the complexity of the algorithm. We also make another extension over CRAP by introducing an improved version of the conditional rotation prediction task. Specifically, we require the rotation prediction branch to predict rotation angle for a mixed version of the rotated image, that is, we randomly mix the input image x i with another randomly sampled rotated image x j via x mix = αx i + (1 − α)x j, with α sampled from [0.5, 1]. Meanwhile, the class prediction p(y|x i ; θ) is calculated from the unmixed version of the input x i. In such a design, the network needs to recognize the rotation angle of the target object with the noisy distraction from another image, and we call this scheme denoising rotation prediction. The purpose of introducing this modified task is to make the SlfSL task more challenging and more dependent on the correct prediction from p(y|x; θ). To see this point, let's consider the following example. Letter'A' is rotated with 270 • and is mixed with letter'B' with rotation 90 •. Directly predicting the rotation angle for this mixed image encounters an ambiguity: whose rotation angle, A's or B's, is the right answer? In other words, the network cannot know which image class is the class-of-interest. This ambiguity can only be resolved from the output of p(y|x; θ) since its input is unmixed target image. Therefore, this improved rotation prediction task relies more on the correct prediction from the semantic classifier and training through CRAP is expected to give stronger supervision signal to p(y|x; θ). Note that although the denoising rotation prediction also uses mix operation, it is completely different from mixup. The latter constructs a loss to require the output of the mixed image to be mixed version of the outputs of original images. This loss is not applied in our method. For more algorithm details about CRAP and the extended CRAP, please refer to the Appendix A.1. In this section, we conduct experiments to evaluate the proposed CRAP method 1. The purpose of our experiments is threefolds: to validate if CRAP is better than other SlfSL-based SemSL algorithms. to compare CRAP and extended CRAP (denoted as CRAP+ hereafter) against the state-of-the-art SemSL methods. to understand the contribution of various components in CRAP. To make a fair comparison to recent works, different experimental protocols are adopted for different datasets. Specifically, for CIFAR-10 and CIFAR-100 and SVHN , we directly follow the settings in . For ILSVRC-2012 (, our settings are identical to except for data pre-processing operations for which we only use the inception crop augmentation and horizontal mirroring. We ensure that all the baselines are compared under the same setting. Followed the standard settings of SemSL, the performance with different amount of labeled samples are tested. For CIFAR-10 and SVHN, sample size of labeled images is ranged in five levels: {250, 500, 1000, 2000, 4000}. For CIFAR-100, 10000 labeled data is used for training. For ILSVRC-2012, 10% and 1% of images are labeled among the whole dataset. In each experiment, three independent trials are conducted for all datasets except for ILSVRC-2012. See more details in Table 8 in Appendix. Firstly, we compare CRAP to other SlfSL-based SemSL algorithms on five datasets: CIFAR-10, CIFAR-100, SVHN, SVHN+Extra and ILSVRC-2012. Two SlfSL-based SemSL baseline approaches are considered: 1) Fine-tune: taking the model pretrained on the pretext task as an initialization and fine-tuning with a set of labeled data. We term this method Fine-tune in the following sections. 2) S 4 L: S 4 L method proposed in. Note that we do not include any methods which combine other SemSL techniques. For this reason, we only use our basic CRAP algorithm in the comparison in this subsection. As a reference, we also report the performance obtained by only using the labeled part of the dataset for training, denoting as Labeled-only. The experimental are as follows: The are presented in Table 1. We find that the "Fine-tune" strategy leads to a mixed amount of improvement over the "Labeled-only" case. It is observed that a large improvement can be obtained when the amount of labeled samples is ranged from 500 to 2000 but not on 250 and 4000's settings. It might be because on one hand too few labeled samples are not sufficient to perform an effective fine-tuning while on the other hand the significant improvement diminishes after the sample size increase. In comparison, S 4 L achieves much better accuracy for the case of using few samples. This is largely benefited from its down-stream-task awareness design -the labeled training samples exerts impact at the feature learning stage. Our CRAP method achieves significantly better performance than those two ways of incorporating SlfSL for SemSL and always halves the test error of S 4 L in most cases. Table 2 shows the of each method. Somehow surprisingly, we find that the Fine-tune and S 4 L do not necessarily outperform the Labeled-only baseline. They actually performs worse than Labeled-only on SVHN. With more training data in SVHN + Extra, S 4 L tends to bring benefits for enhancing performance when the size of labeled samples are small e.g., with 250 samples. In comparison, the proposed CRAP still manages to produce significant improvement over Labeled-only in all those settings. This clearly demonstrates that the simple combination of SlfSL and SemSL may not necessarily bring improvement and a properly-designed strategy of incorporating SlfSL with SemSL is crucial. CIFAR-100 As shown in Table 3, it is obvious that all SlfSL-based SemSL methods can have better accuracy than that of Labeled-only. S 4 L leads to a marginal improvement over Fine-tune although its performance is a little bit unstable on different partitions as shown by its higher variance. Again, the proposed CRAP achieves significant improvement over those baselines. Table 4 presents the of each method. The top block of Table 4 shows the reported in the original S 4 L paper and we also re-implement S 4 L based on the code of. Due to the difference of data pre-processing, in the upper block cannot be directly compared to those below. Again, we have observed that CRAP is consistently superior to S 4 L in all settings. As mentioned in Section 4, for saving the computational cost, we propose to reduce the dimensionality of features fed into the rotation angle predictor when there is a large number of classes. In Table 5, we demonstrates the effect of this scheme. As seen, the test performance stays the same when the feature dimensions is gradually reduced from 2048 to only 16 dimensions. This clearly validates the effectiveness of the proposed scheme. In the following section, we proceed to demonstrate the performance of CRAP+, that is, the extended CRAP method by incorporating the two extensions discussed in Section 4.1 and 4.2. We compare its performance against the current state-of-the-art methods in SemSL. Similar to , several SemSL baselines are considered: Pseudo-Label, Π-Model, Mean Teacher, Virtual Adversarial Training (VAT), MixUp and MixMatch 2. Since a fair and comprehensive comparison has been done in and we strictly follow the same experimental setting, we directly compare CRAP+ to the numbers reported in . The experimental are shown in Figure 2, Figure 3 and Table 6. As seen from those Figures and Table, the proposed CRAP+ is on-par with the best performed approaches, e.g., Mixmatch, in those datasets. This clearly demonstrates the power of the proposed method. Note that the current state-ofthe-art in SemSL is achieved by carefully combining multiple existing successful ideas in SemSL. In contrast, our CRAP+ achieves excellent performance via an innovative framework of marrying SlfSL with SemSL. Conceptually, the latter enjoys greater potential. In fact, CRAP might be further extended by using more successful techniques in SemSL, such as MixUp. Since the focus of this paper is to study how SlfSL can benefit SemSL, we do not pursue this direction here. Since there are several components in CRAP and CRAP+, we study the effect of adding or removing some components in order to provide additional insight into the role of each part. Specifically, we measure the effect of only adding extension 1 to CRAP, i.e., incorporating an additional SemSL loss through sharpening operations on the semantic classifier in auxiliary branches further adding extension 2 to CRAP. The ed model is identical to CRAP+ removing semantic classifier of main branch from CRAP. This is equivalent to using the semantic classifier in auxiliary branches for testing removing rotation angle prediction branch from auxiliary branches and adding extension 1 to CRAP. The ed structure can be seen as a variant of only using the SemSL technique in Extension 1 (but also with the classifier in main branch) removing whole auxiliary branches from CRAP, i.e., pure supervised method with data rotated. We conduct ablation studies on CIFAR-10 with 250 and 4000 labels with presented in Table 7. The main observations are: The two extensions in CRAP+ will bring varying degrees of improvement. Extension 1 in Section 4.1, i.e., a stronger p(y|x; θ) modeling, perhaps leads to greater improvement. Using an additional semantic classifier leads to a slight performance improvement over the strategy of directly utilizing p(y|x; θ) in the auxiliary branches for testing (method in third line from the bottom). Using the sharpening strategy as in our extension 1 and training a SemSL method alone does not produce good performance. This indicates the superior performance of CRAP+ is not simply coming from a strong SemSL method but its incorporation with the CRAP framework. Applying rotation as a data augmentation for labeled data (the last method in Table 7) will not lead to improved performance over the labeled-only baseline as by cross referring the in Table 9. This shows that the advantage of CRAP is not coming from the rotation data augmentation. In this work, we introduce a framework for effectively coupling SemSL with SlfSL. The proposed CRAP method is an implementation of this framework and it shows compelling performance on several benchmark datasets compared to other SlfSL-based SemSL methods. Furthermore, two extensions are incorporated into CRAP to create an improved method which achieves comparable performance to the state-of-the-art SemSL methods.' and should be close to 'Coupling semi-supervised learning with self-supervised learning and explicitly modeling the self-supervised task conditioned on the semi-supervised one' but very far from 'We study whether and how adaptive data augmentation and knowledge distillation can be leveraged simultaneously in a synergistic manner for better training student networks.'

Question: Existing public face image datasets are strongly biased toward Caucasian faces, and other races (e.g., Latino) are significantly underrepresented. The models trained from such datasets suffer from inconsistent classification accuracy, which limits the applicability of face analytic systems to non-White race groups. To mitigate the race bias problem in these datasets, we constructed a novel face image dataset containing 108,501 images which is balanced on race. We define 7 race groups: White, Black, Indian, East Asian, Southeast Asian, Middle Eastern, and Latino. Images were collected from the YFCC-100M Flickr dataset and labeled with race, gender, and age groups. Evaluations were performed on existing face attribute datasets as well as novel image datasets to measure the generalization performance. We find that the model trained from our dataset is substantially more accurate on novel datasets and the accuracy is consistent across race and gender groups. We also compare several commercial computer vision APIs and report their balanced accuracy across gender, race, and age groups. To date, numerous large scale face image datasets (; ; ; ; ; ; ; ; ; ; ;) have been proposed and fostered research and development for automated face detection (b;), alignment (Xiong & De la ;), recognition , generation (; ; ;), modification (; ;), and attribute classification . These systems have been successfully translated into many areas including security, medicine, education, and social sciences. Despite the sheer amount of available data, existing public face datasets are strongly biased toward Caucasian faces, and other races (e.g., Latino) are significantly underrepresented. A recent study shows that most existing large scale face databases are biased towards "lighter skin" faces (around 80%), e.g. White, compared to "darker" faces, e.g. Black . This means the model may not apply to some subpopulations and its may not be compared across different groups without calibration. Biased data will produce biased models trained from it. This will raise ethical concerns about fairness of automated systems, which has emerged as a critical topic of study in the recent machine learning and AI literature . For example, several commercial computer vision systems (Microsoft, IBM, Face++) have been criticized due to their asymmetric accuracy across sub-demographics in recent studies . These studies found that the commercial face gender classification systems all perform better on male and on light faces. This can be caused by the biases in their training data. Various unwanted biases in image datasets can easily occur due to biased selection, capture, and negative sets . Most public large scale face datasets have been collected from popular online media -newspapers, Wikipedia, or web search-and these platforms are more frequently used by or showing White people. To mitigate the race bias in the existing face datasets, we propose a novel face dataset with an emphasis on balanced race composition. Our dataset contains 108,501 facial images collected primarily from the YFCC-100M Flickr dataset (Thomee et al.), which can be freely shared for a research purpose, and also includes examples from other sources such as Twitter and online newspaper outlets. We define 7 race groups: White, Black, Indian, East Asian, Southeast Asian, Middle Eastern, and Latino. Our dataset is well-balanced on these 7 groups (See Figures 1 and 2) Our paper makes three main contributions. First, we emprically show that existing face attribute datasets and models learned from them do not generalize well to unseen data in which more nonWhite faces are present. Second, we show that our new dataset performs better on novel data, not only on average, but also across racial groups, i.e. more consistently. Third, to the best of our knowledge, our dataset is the first large scale face attribute dataset in the wild which includes Latino and Middle Eastern and differentiates East Asian and Southeast Asian. Computer vision has been rapidly transferred into other fields such as economics or social sciences, where researchers want to analyze different demographics using image data. The inclusion of major racial groups, which have been missing in existing datasets, therefore significantly enlarges the applicability of computer vision methods to these fields. The goal of face attribute recognition is to classify various human attributes such as gender, race, age, emotions, expressions or other facial traits from facial appearance (; ;). Table 1 summarizes the statistics of existing large scale public and in-the-wild face attribute datasets including our new dataset. As stated earlier, most of these datasets were constructed from online sources and are typically dominated by the White race. Face attribute recognition has been applied as a sub-component to other computer vision tasks such as face verification and person re-idenfication (; a;). It is imperative to ensure that these systems perform evenly well on different gender and race groups. Failing to do so can be detrimental to the reputations of individual service providers and the public trust about the machine learning and computer vision research community. Most notable incidents regarding the racial bias include Google Photos recognizing African American faces as Gorilla and Nikon's digital cameras prompting a message asking "did someone blink?" to Asian users . These incidents, regardless of whether the models were trained improperly or how much they actually affected the users, often in the termination of the service or features (e.g. dropping sensitive output categories). For this reason, most commercial service providers have stopped providing a race classifier. Face attribute recognition is also used for demographic surveys performed in marketing or social science research, aimed at understanding human social behaviors and their relations to demographic s of individuals. Using off-the-shelf tools and commercial services, social scientists have begun to use images of people to infer their demographic attributes and analyze their behaviors. Notable examples are demographic analyses of social media users using their photographs (; ; ; ;). The cost of unfair classification is huge as it can over-or under-estimate specific sub-populations in their analysis, which may have policy implications. AI and machine learning communities have increasingly paid attention to algorithmic fairness and dataset and model biases (; ; ; . There exist many different definitions of fairness used in the literature . In this paper, we focus on balanced accuracy-whether the attribute classification accuracy is independent of race and gender. More generally, research in fairness is concerned with a model's ability to produce fair outcomes (e.g. loan approval) independent of protected or sensitive attributes such as race or gender. Studies in algorithmic fairness have focused on either 1) discovering (auditing) existing bias in datasets or systems (; ; ;), 2) making a better dataset , or 3) designing a better algorithm or model (; ; ; ;). Our paper falls into the first two categories. The main task of interest in our paper is (balanced) gender classification from facial images. demonstrated many commercial gender classification systems are biased and least accurate on dark-skinned females. The biased may be caused by biased datasets, such as skewed image origins (45% of images are from the U.S. in Imagenet) or biased underlying associations between scene and race in images . It is, however, "infeasible to balance across all possible co-occurrences" of attributes , except in a lab-controlled setting. Therefore, the contribution of our paper is to mitigate, not entirely solve, the current limitations and biases of existing databases by collecting more diverse face images from non-White race groups. We empirically show this significantly improves the generalization performance to novel image datasets whose racial compositions are not dominated by the White race. Furthermore, as shown in Table 1, our dataset is the first large scale in-the-wild face image dataset which includes Southeast Asian and Middle Eastern races. While their faces share similarity with East Asian and White groups, we argue that not having these major race groups in datasets is a strong form of discrimination. 3 DATASET CONSTRUCTION 3.1 RACE TAXONOMY Our dataset defines 7 race groups: White, Black, Indian, East Asian, Southeast Asian, Middle Eastern, and Latino. Race and ethnicity are different categorizations of humans. Race is defined based on physical traits and ethnicity is based on cultural similarities . For example, Asian immigrants in Latin America can be of Latino ethnicity. In practice, these two terms are often used interchangeably. We first adopted a commonly accepted race classification from the U.S. Census Bureau (White, Black, Asian, Hawaiian and Pacific Islanders, Native Americans, and Latino). Latino is often treated as an ethnicity, but we consider Latino a race, which can be judged from the facial appearance. We then further divided subgroups such as Middle Eastern, East Asian, Southeast Asian, and Indian, as they look clearly distinct. During the data collection, we found very few examples for Hawaiian and Pacific Islanders and Native Americans and discarded these categories. All the experiments conducted in this paper were therefore based on 7 race classification. An important criterion to measure dataset bias is on which basis the bias should be measured: skin color or race? A few recent studies use skin color as a proxy to racial or ethnicity grouping. While skin color can be easily computed without subjective annotations, it has limitations. First, skin color is heavily affected by illumination and light conditions. The Pilot Parliaments Benchmark (PPB) dataset only used profile photographs of government officials taken in well controlled lighting, which makes it non-in-the-wild. Second, within-group variations of skin color are huge. Even same individuals can show different skin colors over time. Third, most importantly, race is a multidimensional concept whereas skin color (i.e. brightness) is one dimensional. Figure 5 in Appendix shows the distributions of the skin color of multiple race groups, measured by Individual Typology Angle (ITA) . As shown here, the skin color provides no information to differentiate many groups such as East Asian and White. Therefore, we explicitly use race and annotate the physical race by human annotators' judgments. To complement the limits of race categorization, however, we also use skin color, measured by ITA, following the same procedure used by. Many existing face datasets have been sourced from photographs of public figures such as politicians or celebrities (; ; ; ;). Despite the easiness of collecting images and ground truth attributes, the selection of these populations may be biased. For example, politicians may be older and actors may be more attractive than typical faces. Their images are usually taken by professional photographers in limited situations, leading to the quality bias. Some datasets were collected via web search using keywords such as "Asian boy" . These queries may return only stereotypical faces or prioritize celebrities in those categories rather than diverse individuals among general public. Our goal is to minimize the selection bias introduced by such filtering and maximize the diversity and coverage of the dataset. We started from a huge public image dataset, Yahoo YFCC100M dataset (Thomee et al.), and detected faces from the images without any preselection. A recent work also used the same dataset to construct a huge unfiltered face dataset (Diversity in Faces, DiF) . Our dataset is smaller but more balanced on race (See Figure 1). For an efficient collection, we incrementally increased the dataset size. We first detected and annotated 7,125 faces randomly sampled from the entire YFCC100M dataset ignoring the locations of images. After obtaining annotations on this initial set, we estimated demographic compositions of each country. Based on this statistic, we adaptively adjusted the number of images for each country sampled from the dataset such that the dataset is not dominated by the White race. Consequently, we excluded the U.S. and European countries in the later stage of data collection after we sampled enough White faces from those countries. The minimum size of a detected face was set to 50 by 50 pixels. This is a relatively smaller size compared to other datasets, but we find the attributes are still recognizable and these examples can actually make the classifiers more robust against noisy data. We only used images with "Attribution" and "Share Alike" Creative Commons licenses, which allow derivative work and commercial usages. We used Amazon Mechanical Turk to annotate the race, gender and age group for each face. We assigned three workers for each image. If two or three workers agreed on their judgements, we took the values as ground-truth. If all three workers produced different responses, we republished the image to another 3 workers and subsequently discarded the image if the new annotators did not agree. These annotations at this stage were still noisy. We further refined the annotations by training a model from the initial ground truth annotations and applying back to the dataset. We then manually re-verified the annotations for images whose annotations differed from model predictions. We first measure how skewed each dataset is in terms of its race composition. For the datasets with race annotations, we use the reported statistics. For the other datasets, we annotated the race labels for 3,000 random samples drawn from each dataset. See Figure 1 for the . As expected, most existing face attribute datasets, especially the ones focusing on celebrities or politicians, are biased toward the White race. Unlike race, we find that most datasets are relatively more balanced on gender ranging from 40%-60% male ratio. To compare model performance of different datasets, we used an identical model architecture, ResNet-34, to be trained from each dataset. We used ADAM optimization with a learning rate of 0.0001. Given an image, we detected faces using the dlib's (dlib.net) CNN-based face detector and ran the attribute classifier on each face. The experiment was done in PyTorch. Throughout the evaluations, we compare our dataset with three other datasets: UTKFace , LFWA+, and CelebA . Both UTKFace and LFWA+ have race annotations, and thus, are suitable for comparison with our dataset. CelebA does not have race annotations, so we only use it for gender classification. See Table 1 for more detailed dataset characteristics..971 --* CelebA doesn't provide race annotations. The was obtained from the whole set (white and non-white). † FairFace defines 7 race categories but only 4 races (White, Black, Asian, and Indian) were used in this to make it comparable to UTKFace. Using models trained from these datasets, we first performed cross-dataset classifications, by alternating training sets and test sets. Note that FairFace is the only dataset with 7 races. To make it compatible with other datasets, we merged our fine racial groups when tested on other datasets. CelebA does not have race annotations but was included for gender classification. Tables 2 and 3 show the classification for race, gender, and age on the datasets across subpopulations. As expected, each model tends to perform better on the same dataset on which it was trained. However, the accuracy of our model was highest on some variables on the LFWA+ dataset and also very close to the leader in other cases. This is partly because LFWA+ is the most biased dataset and ours is the most diverse, and thus more generalizable dataset. To test the generalization performance of the models, we consider three novel datasets. Note that these datasets were collected from completely different sources than our data from Flickr and not used in training. Since we want to measure the effectiveness of the model on diverse races, we chose the test datasets that contain people in different locations as follows. Geo-tagged Tweets. First we consider images uploaded by Twitter users whose locations are identified by geo-tags (longitude and latitude), provided by . From this set, we chose four countries (France, Iraq, Philippines, and Venezuela) and randomly sampled 5,000 faces. Media Photographs. Next, we also use photographs posted by 500 online professional media outlets. Specifically, we use a public dataset of tweet IDs posted by 4,000 known media accounts, e.g. @nytimes. Note that although we use Twitter to access the photographs, these tweets are simply external links to pages in the main newspaper sites. Therefore this data is considered as media photographs and different from general tweet images mostly uploaded by ordinary users. We randomly sampled 8,000 faces from the set. Protest Dataset. Lastly, we also use a public image dataset collected for a recent protest activity study . The authors collected the majority of data from Google Image search by using keywords such as "Venezuela protest" or "football game" (for hard negatives). The dataset exhibits a wide range of diverse race and gender groups engaging in different activities in various countries. We randomly sampled 8,000 faces from the set. These faces were annotated for gender, race, and age by Amazon Mechanical Turk workers. Table 4: Gender classification accuracy measured on external validation datasets across gender-race groups. Table 7 shows the classification accuracy of different models. Because our dataset is larger than LFWA+ and UTKFace, we report the three variants of the FairFace model by limiting the size of a training set (9k, 18k, and Full) for fair comparisons. Improved Accuracy. As clearly shown in the , the model trained by FairFace outperforms all the other models for race, gender, and age, on the novel datasets, which have never been used in training and also come from different data sources. The models trained with fewer training images (9k and 18k) still outperform other datasets including CelebA which is larger than FairFace. This suggests that the dataset size is not the only reason for the performance improvement. Balanced Accuracy. Our model also produces more consistent -for race, gender, age classification -across different race groups compared to other datasets. We measure the model consistency by standard deviations of classification accuracy measured on different sub-populations, as shown in Table 5. More formally, one can consider conditional use accuracy equality (Berk et al.) or equalized odds as the measure of fair classification. For gender classification: where Y is the predicted gender, Y is the true gender, A refers to the demographic group, and D is the set of different demographic groups being considered (race). When we consider different gender groups for A, this needs to be modified to measure accuracy equality Berk et al.: We therefore define the maximum accuracy disparity of a classifier as follows: Table 4 shows the gender classification accuracy of different models measured on the external validation datasets for each race and gender group. The FairFace model achieves the lowest maximum accuracy disparity. The LFWA+ model yields the highest disparity, strongly biased toward the male category. The CelebA model tends to exhibit a bias toward the female category as the dataset contains more female images than male. The FairFace model achieves less than 1% accuracy discrepancy between male ↔ female and White ↔ non-White for gender classification (Table 7). All the other models show a strong bias toward the male class, yielding much lower accuracy on the female group, and perform more inaccurately on the non-White group. The gender performance gap was the biggest in LFWA+ (32%), which is the smallest among the datasets used in the experiment. Recent work has also reported asymmetric gender biases in commercial computer vision services , and our further suggests the cause is likely due to the unbalanced representation in training data. Data Coverage and Diversity. We further investigate dataset characteristics to measure the data diversity in our dataset. We first visualize randomly sampled faces in 2D space using t-SNE as shown in Figure 3. We used the facial embedding based on ResNet-34 from dlib, which was trained from the FaceScrub dataset , the VGG-Face dataset and other online sources, which are likely dominated by the White faces. The faces in FairFace are well spread in the space, and the race groups are loosely separated from each other. This is in part because the embedding was trained from biased datasets, but it also suggests that the dataset contains many non-typical examples. LFWA+ was derived from LFW, which was developed for face recognition, and therefore contains multiple images of the same individuals, i.e. clusters. UTKFace also tends to focus more on local clusters compared to FairFace. To explicitly measure the diversity of faces in these datasets, we examine the distributions of pairwise distance between faces (Figure 4). On the random subsets, we first obtained the same 128-dimensional facial embedding from dlib and measured pair-wise distance. Figure 4 shows the CDF functions for 3 datasets. As conjectured, UTKFace had more faces that are tightly clustered together and very similar to each other, compared to our dataset. Surprisingly, the faces in LFWA+ were shown very diverse and far from each other, even though the majority of the examples contained a white face. We believe this is mostly due to the fact that the face embedding was also trained on a very similar white-oriented dataset which will be effective in separating white faces, not because the appearance of their faces is actually diverse. (See Figure 2) Figure 4: Distribution of pairwise distances of faces in 3 datasets measured by L1 distance on face embedding. Previous studies have reported that popular commercial face analytic models show inconsistent classification accuracies across different demographic groups . We used the FairFace images to test several online APIs for gender classification: Microsoft Face API, Amazon Rekognition, IBM Watson Visual Recognition, and Face++. Compared to prior work using politicians' faces, our dataset is much more diverse in terms of race, age, expressions, head orientation, and photographic conditions, and thus serves as a much better benchmark for bias measurement. We used 7,476 random samples from FairFace such that it contains an equal number of faces from each race, gender, and age group. We left out children under the age of 20, as these pictures were often ambiguous and the gender could not be determined for certain. The experiments were conducted on August 13th -16th, 2019..923.966.901.955.925.949.918.914.921.987.951.979.906.983.941.030 Microsoft.822.777.766.717.824.775.852.794.843.848.863.790.839.772.806.042 Face++.888.959.805.944.876.904.884.897.865.981.770.968.822.978.896.066 IBM.910.966.758.927.899.910.852.919.884.972.811.957.871.959.900.061 FairFace.987.991.964.974.966.979.978.961.991.989.991.987.972.991.980.011 *Microsoft.973.998.962.967.963.976.960.957.983.993.975.991.966.993.975.014 *Face++.893.968.810.956.878.911.886.899.870.983.773.975.827.983.901.067 *IBM.914.981.761.956.909.920.852.926.892.977.819.975.881.979.910.066 Table 6 shows the gender classification accuracies of the tested APIs. These APIs first detect a face from an input image and classify its gender. Not all 7,476 faces were detected by these APIs with the exception of Amazon Rekognition which detected all of them. Table 8 in Appendix reports the detection rate. 1 We report two sets of accuracies: 1) treating mis-detections as mis-classifications and 2) excluding mis-detections. For comparison, we included a model trained with our dataset to provide an upper bound for classification accuracy. Following prior work , we also show the classification accuracy as a function of skin color in Figure 6. The suggest several findings. First, all tested gender classifiers still favor the male category, which is consistent with the previous report . Second, dark-skinned females tend to yield higher classification error rates, but there exist many exceptions. For example, Indians have darker skin tones (Figure 5), but some APIs (Amazon and MS) classified them more accurately than Whites. This suggests skin color alone, or any other individual phenotypic feature, is not a sufficient guideline to study model bias. Third, face detection can also introduce significant gender bias. Microsoft's model failed to detect many male faces, an opposite direction from the gender classification bias. This was not reported in previous studies which only used clean profile images of frontal faces. This paper proposes a novel face image dataset balanced on race, gender and age. Compared to existing large-scale in-the-wild datasets, our dataset achieves much better generalization classification performance for gender, race, and age on novel image datasets collected from Twitter, international online newspapers, and web search, which contain more non-White faces than typical face datasets. We show that the model trained from our dataset produces balanced accuracy across race, whereas other datasets often lead to asymmetric accuracy on different race groups. This dataset was derived from the Yahoo YFCC100m dataset (Thomee et al.) for the images with Creative Common Licenses by Attribution and Share Alike, which permit both academic and commercial usage. Our dataset can be used for training a new model and verifying balanced accuracy of existing classifiers. Algorithmic fairness is an important aspect to consider in designing and developing AI systems, especially because these systems are being translated into many areas in our society and affecting our decision making. Large scale image datasets have contributed to the recent success in computer vision by improving model accuracy; yet the public and media have doubts about its transparency. The novel dataset proposed in this paper will help us discover and mitigate race and gender bias present in computer vision systems such that such systems can be more easily accepted in society. A APPENDIX Figure 5: Individual Typology Angle (ITA), i.e. skin color, distribution of different races measured in our dataset.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

A new face image dataset for balanced race, gender, and age which can be used for bias measurement and mitigation

represent
For example, Achieving strong adversarial robustness comparable to adversarial training without training on adversarial examples should be similar to Adversarial training is one of the strongest defenses against adversarial attacks, but it requires adversarial examples to be generated for every mini-batch during optimization. The expense of producing these examples during training often precludes adversarial training from use on complex image datasets. In this study, we explore the mechanisms by which adversarial training improves classifier robustness, and show that these mechanisms can be effectively mimicked using simple regularization methods, including label smoothing and logit squeezing. Remarkably, using these simple regularization methods in combination with Gaussian noise injection, we are able to achieve strong adversarial robustness -- often exceeding that of adversarial training -- using no adversarial examples. Deep Neural Networks (DNNs) have enjoyed great success in many areas of computer vision, such as classification BID7, object detection BID4, and face recognition BID11. However, the existence of adversarial examples has raised concerns about the security of computer vision systems BID16 BID1. For example, an attacker may cause a system to mistake a stop sign for another object BID3 or mistake one person for another BID14. To address security concerns for high-stakes applications, researchers are searching for ways to make models more robust to attacks. Many defenses have been proposed to combat adversarial examples. Approaches such as feature squeezing, denoising, and encoding BID19 BID13 BID15 BID10 have had some success at pre-processing images to remove adversarial perturbations. Other approaches focus on hardening neural classifiers to reduce adversarial susceptibility. This includes specialized non-linearities BID20, modified training processes BID12, and gradient obfuscation BID0.Despite all of these innovations, adversarial training BID5, one of the earliest defenses, still remains among the most effective and popular strategies. In its simplest form, adversarial training minimizes a loss function that measures performance of the model on both clean and adversarial data as follows DISPLAYFORM0 where L is a standard (cross entropy) loss function, (x i, y i) is an input image/label pair, θ contains the classifier's trainable parameters, κ is a hyper-parameter, and x i,adv is an adversarial example for image x. BID9 pose adversarial training as a game between two players that similarly requires computing adversarial examples on each iteration. A key drawback to adversarial training methods is their computational cost; after every mini-batch of training data is formed, a batch of adversarial examples must be produced. To train a network that resists strong attacks, one needs to train with the strongest adversarial examples possible. For example, networks hardened against the inexpensive Fast Gradient Sign Method can be broken by a simple two-stage attack BID17. Current state-of-theart adversarial training on MNIST and CIFAR-10 use expensive iterative adversaries BID9, such as the Projected Gradient Descent (PGD) method, or the closely related Basic Iterative Method (BIM) BID8. Adversarial training using strong attacks may be 10-100 times more time consuming than standard training methods. This prohibitive cost makes it difficult to scale adversarial training to larger datasets and higher resolutions. In this study, we show that it is possible to achieve strong robustness -comparable to or greater than the robustness of adversarial training with a strong iterative attack -using fast optimization without adversarial examples. We achieve this using standard regularization methods, such as label smoothing BID18 and the more recently proposed logit squeezing BID6. While it has been known for some time that these tricks can improve the robustness of models, we observe that an aggressive application of these inexpensive tricks, combined with random Gaussian noise, are enough to match or even surpass the performance of adversarial training on some datasets. For example, using only label smoothing and augmentation with random Gaussian noise, we produce a CIFAR-10 classifier that achieves over 73% accuracy against black-box iterative attacks, compared to 64% for a state-of-the-art adversarially trained classifier BID9. In the white-box case, classifiers trained with logit squeezing and label smoothing get ≈ 50% accuracy on iterative attacks in comparison to ≈ 47% for adversarial training. Regularized networks without adversarial training are also more robust against non-iterative attacks, and more accurate on non-adversarial examples. Our goal is not just to demonstrate these defenses, but also to dig deep into what adversarial training does, and how it compares to less expensive regularization-based defenses. We begin by dissecting adversarial training, and examining ways in which it achieves robustness. We then discuss label smoothing and logit squeezing regularizers, and how their effects compare to those of adversarial training. We then turn our attention to random Gaussian data augmentation, and explore the importance of this technique for adversarial robustness. Finally, we combine the regularization methods with random Gaussian augmentation, and experimentally compare the robustness achievable using these simple methods to that achievable using adversarial training. Adversarial training injects adversarial examples into the training data as SGD runs. During training, adversarial perturbations are applied to each training image to decrease the logit corresponding to its correct class. The network must learn to produce logit representations that preserve the correct labeling even when faced with such an attack. At first glance, it seems that adversarial training might work by producing a large "logit gap," i.e., by producing a logit for the true class that is much larger than the logit of other classes. Surprisingly, adversarial training has the opposite effect -we will see below that it decreases the logit gap. To better understand what adversarial training does, and how we can replicate it, we now break down the different strategies for achieving robustness. This section presents a simple metric for adversarial robustness that will help us understand adversarial training. Consider an image x, and its logit representation z (i.e. pre-softmax activation) produced by a neural network. Let z y denote the logit corresponding to the correct class y. If we add a small perturbation δ to x, then the corresponding change in logits is approximately δ T ∇ x z y under a linear approximation, where ∇ x z y is the gradient of z y with respect to x. Under a linearity assumption, we can calculate the step-size L needed to move an example from class y to another classȳ. A classifier is susceptible to adversarial perturbation δ if the perturbed logit of the true class is smaller than the perturbed logit of any other class: DISPLAYFORM0 Assuming a one-step ∞ attack such as FGSM, the perturbation δ can be expressed as DISPLAYFORM1 where L is the ∞ -norm of the perturbation. Using this choice of δ, Equation 2 becomes Table 1: Experimental and predicted accuracy of classifiers for MNIST and CIFAR-10. The predicted accuracy is the percentage of images for which < L. The empirical accuracy is the percent of images that survive a perturbation of size. Attacks on both the cross-entropy (X-ent) and logits as in BID2 (CW) are presented. DISPLAYFORM2 where · 1 denotes the 1 -norm of a vector. Therefore the smallest ∞ -norm of the perturbation required is the ratio of "logit gap" to "gradient gap", i.e., DISPLAYFORM3 Equation FORMULA4 measures robustness by predicting the smallest perturbation size needed to switch the class of an image. While the formula for L makes linearity assumptions, the approximation L fairly accurately predicts the robustness of classifiers of the CIFAR-10 dataset (where perturbations are small and linearity assumptions cause little distortion). It is also a good ballpark approximation on MNIST, even after adversarial training (see Table 1).Maximal robustness occurs when L is as large as possible. From equation 4, we observe 3 different strategies for hardening a classifier:• Increase the logit gap: Maximize the numerator of equation 4 by producing a classifier with relatively large z y.• Squash the adversarial gradients: Train a classifier that has small adversarial gradients ∇ x zȳ for any classȳ. In this case a large perturbation is needed to significantly change zȳ.• Maximize gradient coherence: Produce adversarial gradients ∇ x zȳ that are highly correlated with the gradient for the correct class ∇ x z y. This will shrink the denominator of equation 4, and produce robustness even if adversarial gradients are large. In this case, one cannot decrease z y without also decreasing zȳ, and so large perturbations are needed to change the class label. The most obvious strategy for achieving robustness is to increase the numerator in equation 4 while fixing the denominator. Remarkably, our experimental investigation reveals that adversarial training does not rely on this strategy at all, but rather it decreases the logit gap and gradient gap simultaneously. This can be observed in FIG0, which shows distributions of logit gaps for naturally and adversarially trained models on MNIST. Note that the cross entropy loss actually limits adversarial training from increasing logit gaps. The accuracy of the classifier goes down in the presence of adversarial examples, and so the cross entropy loss is minimized by smaller logit gaps that reflect the lower level of certainty in the adversarial training environment. Adversarial training succeeds by minimizing the denominator in Equation 4; it simultaneously squeezes the logits and crushes the adversarial gradients. FIG0 shows that the adversarial gradients shrink dramatically more than the logit gaps, and so the net effect is an increase in robustness. If we closely examine the phenomenon of shrinking the logit gaps, we find that this shrink is due in part to an overall shrink in the size of the logits themselves, (i.e., |z i | for any class i). To see this, we plot histograms of the logits when classifiers are adversarially trained with strong adversaries 1, weak adversaries 2, and with no adversarial examples. FIG1 shows that adversarial training does indeed squash the logits, although not enough to fully explain the decrease in |z y − zȳ| in FIG0. 3 We have seen that adversarial training works by squashing adversarial gradients and slightly increasing gradient coherence. But the fact that it cannot do this without decreasing the logit gap leads us to suspect that these quantities are inherently linked. This leads us to ask an important question: If we directly decrease the logit gap, or the logits themselves, using an explicit regularization term, will this have the desired effect of crushing the adversarial gradients? There are two approaches to replicating the effect on the logits produced by adversarial training. The first is to replicate the decrease in logit gap seen in FIG0. This can be achieved by label smoothing. A second approach to replicating adversarial training is to just directly crush all logit values and mimic the behavior in FIG1. This approach is known as "logit squeezing," and works by adding a regularization term to the training objective that explicitly penalizes large logits. Label smoothing Label smoothing converts "one-hot" label vectors into "one-warm" vectors that represents a low-confidence classification. Because large logit gaps produce high-confidence classifications, label-smoothed training data forces the classifier to produce small logit gaps. Label smoothing is a commonly used trick to prevent over-fitting on general classification problems, and it was first observed to boost adversarial robustness by BID18, where it was used as an inexpensive replacement for the network distillation defense BID12. A one-hot label vector y hot is smoothed using the formula DISPLAYFORM0 where α ∈ is the smoothing parameter, and N c is the number of classes. If we pick α = 0 we get a hard decision vector with no smoothing, while α = 1 creates an ambiguous decision by assigning equal certainty to all classes. Logit squeezing A second approach to replicating adversarial training is to just directly crush all logit values and mimic the behavior in FIG1. This approach is known as "logit squeezing," and works by adding a regularization term to the training objective that explicitly penalizes large logits. BID6 were the first to introduce logit-squeezing as an alternative to a "logit pairing" defense. Logit squeezing relies on the loss function DISPLAYFORM1 where β is the squeezing parameter (i.e., coefficient for the logit-squeezing term) and ||.|| F is the Frobenius norm of the logits for the mini-batch. Can such simple regularizers really replicate adversarial training? Our experimental suggest that simple regularizers can hurt adversarial robustness, which agrees with the findings in BID20. However, these strategies become highly effective when combined with a simple trick from the adversarial training literature -data augmentation with Gaussian noise. Adding Gaussian noise to images during training (i.e, Gaussian noise augmentation) can be used to improve the adversarial robustness of classifiers BID6 BID20. However, the effect of Gaussian noise is not well understood BID6. Here, we take a closer look at the behavior of Gaussian augmentation through systematic experimental investigations, and see that its effects are more complex than one might think. Label smoothing and logit squeezing become shockingly effective at hardening networks when they are combined with Gaussian noise augmentation. From the robustness plots in FIG2, we can see that training with Gaussian noise alone produces a noticeable change in robustness, which seems to be mostly attributable to a widening of the logit gap and slight decrease in the gradient gap (∇ x z y − ∇ x zȳ 1). The small increase in robustness from random Gaussian augmentation was also reported by BID6. We also see that label smoothing alone causes a very slight drop in robustness; the shrink in the gradient gap is completely offset by a collapse in the logit gap. Surprisingly, Gaussian noise and label smoothing have a powerful synergistic effect. When used together they cause a dramatic drop in the gradient gap, leading to a surge in robustness. A similar effect happens in the case of logit squeezing, and are shown in Appendix B (Figure 8). Regularization methods have the potential to squash or align the adversarial gradients, but these properties are only imposed during training on images from the manifold that the "true" data lies on. At test time, the classifier sees adversarial images that do not "look like" training data because they lie off of, but adjacent to, the image manifold. By training the classifier on images with random perturbations, we teach the classifier to enforce the desired properties for input images that lie off the manifold. The generalization property of Gaussian augmentation seems to be independent from, and sometimes conflicting with, the synergistic properties discuss above. In our experiments below, we find that smaller noise levels lead to a stronger synergistic effect, and yield larger L and better robustness to FGSM attacks. However, larger noise levels enable the regularization properties to generalize further off the manifold, ing in better robustness to iterative attacks or attacks that escape the flattened region by adding an initial random perturbation. See the on MNIST in Table 2 and the on CIFAR-10 in TAB3 for various values of σ (standard deviation of Gaussian noise). For more comprehensive experiments on the different parameters that contribute to the regularizers see Table 6 for MNIST and Tables 7 & 8 Label smoothing (i.e. reducing the variance of the logits) is helpful because it causes the gradient gap to decrease. The decreased gradient gap may be due to smaller element-wise gradient amplitudes, the alignments of the adversarial gradients, or both. To investigate the causes, we plot the 1 norm of the gradients of the logits with respect to the input image 4 and the cosine of the angle between the gradients FIG3. We see that in label smoothing (with Gaussian augmentation), both the gradient magnitude decreases and the gradients get more aligned. Larger smoothing parameters α cause the gradient to be both smaller and more aligned. When logit squeezing is used with Gaussian augmentation, the magnitudes of the gradients decrease. The distribution of the cosines between gradients widens, but does not increase like it did for label smoothing. These effects are very similar to the behavior of adversarial training in FIG0. Interestingly, in the case of logit squeezing with Gaussian noise, unlike label smoothing, the numerator of Equation 4 increases as well. This increase in the logit gap disappears once we take away Gaussian augmentation (See Appendix B Figure 8). Simultaneously increasing the numerator and decreasing the denominator of Equation 4 potentially gives a slight advantage to logit squeezing. There are multiple factors that can affect the robustness of the MNIST classifier 5. While regularizers do not yield more robustness than adversarial training for MNIST, the are promising given that these relatively high values of robustness come at a cheap cost in comparison to adversarial training. In Table 2 we notice that as we increase the number of training iterations k, we get more robust models for both logit squeezing and label smoothing 6. We get more robust models when we use larger smoothing (α) and squeezing (β) parameters, and when Gaussian augmentation is used with standard deviation σ that is greater than the desired (the maximum perturbation size). Table 2: Accuracy of different MNIST classifiers against PGD and FGSM attacks on X-ent and CW losses under the white-box and black-box threat models. Attacks have maximum ∞ perturbation = 0.3. The iterative white-box attacks have an initial random step. The naturally trained model is used for generating black-box attacks. We use CW loss for the black-box attack. We trained Wide-Resnet CIFAR-10 classifiers (depth=28 and k=10) using aggressive values for the smoothing and squeezing parameters on the CIFAR10 data set. Similar to BID9, we use the standard data-augmentation techniques and weight-decay. We compare our to those of BID9. Note that the adversarially trained model from BID9 has been trained for 80,000 iterations on adversaries which are generated using a 7-step PGD. Keeping in mind that each step requires a forward and backward pass, the running time of training for 80,000 iterations on 7-step PDG examples is equivalent to 640,000 iterations of training with label smoothing or logit squeezing. A short version of our on white-box attacks are summarized in TAB3. The of some of our black-box experiments are summarized in Table 4 7. While logit squeezing seems to outperform label smoothing in the white-box setting, label smoothing is slightly better under the black-box threat. We see that aggressive logit squeezing with squeezing parameter β = 10 and σ = 20 in a model that is more robust than the adversarially trained model from BID9 BID9 63.39%* 64.38%* 63.39%* 64.38%* 67.00% 67.25% Table 4: Black-box attacks on CIFAR-10 models. Attacks are ∞ with = 8. Similar to BID9, We build 7-step PGD attacks and FGSM attacks for a public adversarially trained model. *Values taken from the original paper by BID9. Some defenses work by obfuscating gradients and masking the gradients. BID0 suggest these models can be identified by performing "sanity checks" such as attacking them with Figure 5: The cross-entropy landscape of the first eight images of the validation set for the model trained for 160k iterations with hyper-parameters β = 10 and σ = 30. To plot the loss landscape we take two random directions r 1 and r 2 (i.e. r 1, r 2 ∼ Rademacher(0.5) ). We plot the cross-entropy (i.e. xent) loss at different points x = x i + 1 · r 1 + 2 · r 2. Where x i is the clean image and −10 ≤ 1, 2 ≤ 10.unbounded strong adversaries (i.e. unbounded with many iterations). By attacking our robust models using these unbounded attacks, we verify that the unbounded adversary can degrade the accuracy to 0.00% which implies that the adversarial example generation optimization attack (PGD) is working properly. Also, it is known that models which do not completely break the PGD attack (such as us) can possibly lead to a false sense of security by creating a loss landscape that prevents an -bounded but weak adversary from finding strong adversarial examples. This can be done by convoluting the loss landscape such that many local-optimal points exist. This false sense of security can be identified by increasing the strength of the adversary using methods such as increasing the number of steps and random restarts of the PGD attack. We run the stronger PGD attacks on a sample model from TAB3 with hyper-parameters k = 160k, β = 10, and σ = 30. We notice that performing 9 random restarts for the PGD attack on the cross-entropy loss only drops the accuracy slightly to 49.86%. Increasing the number of PGD steps to 1000 decreases the accuracy slightly more to 40.94% 8. While for such strong iterative white-box attacks our sample model is less robust than the adversarially trained model, there are other areas where this hardened model is superior to the adversarially trained model: Very high robustness in the black box setting (roughly 3% higher than that for adversarial training according to Table 4) and against white-box non-iterative (or less iterative) attacks (roughly 15%); high test accuracy on clean data (roughly 3%); and, very fast training time compared to adversarial training. To further verify that our robust model is not falsely creating a sense of robustness by "breaking" the optimization problem that generates adversarial examples by either masking the gradients or making the loss landscape convoluted, we visualize the loss landscape of our sample model from TAB3. We plot the classification (e.g., cross-entropy) loss for points surrounding the first eight validation images that belong to the subspace spanned by two random directions 9 in Figure 5. It seems that the loss landscape has not become convoluted. This observation is backed up by the fact that increasing the number of PGD attack steps does not substantially affect accuracy. To check the performance of our proposed regularizers on more complicated datasets with more number of classes, we perform aggressive logit squeezing on the CIFAR-100 dataset which contains 100 categories. We use the same architecture and settings used for training the CIFAR-10 classifiers. The white-box performance of two hardened models with logit squeezing and a PGD adversarially trained model for the same architecture are summarized in Table 5 Table 5: White-box iterative attacks on CIFAR-100 models. We use ∞ attacks with = 8. For brevity, we only report the for attacking the cross-entropy loss. We attack the models with adversaries having different strengths by varying the number of PGD steps. Similar to CIFAR-10, aggressive logit squeezing can in models as robust as those that are adversarially trained at a fraction of the cost. The logit-squeezed model that is trained for only 80k iterations achieves high classification accuracy for natural/clean examples. It is also more robust against white-box PGD attacks compared to the adversarially trained model that requires much more training time. The logit-squeezed model with k = 160k improves the robustness and clean accuracy even further and still trains faster than the adversarially trained model (28.8 hours vs 34.3 hours). We studied the robustness of adversarial training, label smoothing, and logit squeezing through a linear approximation L that relates the magnitude of adversarial perturbations to the logit gap and the difference between the adversarial directions for different labels. Using this simple model, we observe how adversarial training achieves robustness and try to imitate this robustness using label smoothing and logit squeezing. The ing methods perform well on MNIST, and can get on CIFAR-10 and CIFAR-100 that can excel over adversarial training in both robustness and accuracy on clean examples. By demonstrating the effectiveness of these simple regularization methods, we hope this work can help make robust training easier and more accessible to practitioners. Similarly to what we observed about adversarial training on MNIST, adversarial training on CIFAR-10 works by greatly shrinking the adversarial gradients and also shrinking the logit gaps. The shrink in the gradients is much more dramatic than the shrink in the logit gap, and overwhelms the decrease in the numerator of Equation 4. See FIG4. As shown empirically in Table 6, and analytically using the linear approximation in Equation 4 evaluated in Figure 8, logit squeezing worsens robustness when Gaussian augmentation is not used. However, when fused with Gaussian augmentation, logit squeezing achieves good levels of robustness. This addition of Gaussian augmentation has three observed effects: the gradients get squashed, the logit gap increases, and the gradients get slightly more aligned. The increase in the logit gaps increases the numerator in Equation 4. This gives a slight edge to logit squeezing in comparison to label smoothing, that mostly works by decreasing the denominator in Equation 4. The for all of our experiments on MNIST are summarized in Table 6. As can be seen, Gaussian random augmentation by itself (β = α = 0) is effective in increasing robustness on black-box attacks. It does not, however, significantly increase robustness on white-box attacks. Models that are only trained with either logit squeezing or label smoothing without random Gaussian augmentation (σ = 0), can be fooled by an adversary that has knowledge of the model parameters (white-box) with accuracy 100%. In the black-box setting, they are also not robust. Increasing the magnitude of the noise (σ) generally increases robustness but degrades the performance on the clean examples. Keeping the number of iterations k constant, for extremely large σ Table 6: Accuracy of different models trained on MNIST with a 40 step PGD attack on the crossentropy (X-ent) loss and the Carlini-Wagner (CW) loss under the white-box and black-box threat models. Attacks are ∞ attacks with a maximum perturbation of = 0.3. The iterative white-box attacks have an initial random step. The naturally trained model was used for generating the attacks for the black-box threat model. We use the CW loss for the FGSM attack in the blackbox case. k is the number of training iterations. Here we take a deeper look at reguarlized training for CIFAR-10. The that can be drawn in this case are parallel with those of MNIST discussed in Appendix C. It is worth noting that while the of logit squeezing outperform those from label smoothing in the white-box setting, training with large squeezing coefficient β often fails and in low accuracy on test data. This breakdown of training rarely happens for label smoothing (even for very large smoothing parameters α). BID9 100.00% 87.25% 45.84% 46.90% 56.22% 55.57% Table 7: White-box attacks on the CIFAR-10 models. All attacks are ∞ attacks with = 8. For the 20-step PGD, similar to BID9, we use an initial random perturbation. While it seems that logit squeezing, label smoothing, and adversarial training have a lot in common when we look at quantities affecting the linear approximation L, we wonder whether they still look similar with respect to other metrics. Here, we look at the sum of the activations in the logit layer for every logit FIG0 ) and the sum of activations for every neuron of the penultimate layer (FIG0). The penultimate activations are often seen as the "feature representation" that the neural network learns. By summing over the absolute value of the activations of all test examples for every neuron in the penultimate layer, we can identify how many of the neurons are effectively inactive. When we perform natural training, all neurons become active for at least some images. After adversarial training, this is no longer the case. Adversarial training is causing the effective dimensionality of the deep feature representation layer to decrease. One can interpret this as adversarial training learning to ignore features that the adversary can exploit (400 out of the 1024 neurons of the penultimate layer are deactivated). Shockingly, both label smoothing and logit squeezing do the samethey deactivate roughly 400 neurons from the deep feature representation layer. BID9 63.39%* 64.38%* 63.39%* 64.38%* 67.00% 67.25% Table 8: Black-box attacks on the CIFAR-10 models. All attacks are ∞ attacks with = 8. Similar to BID9, We build 7-step PGD attacks and FGSM attacks for the public adversarial trained model of MadryLab. We then use the built attacks for attacking the different models. *: Since we do not have the Madry model, we cannot evaluate it under the PGD attack with and without random initialization and therefore we use the same value that is reported by them for both. As a sanity check, and to verify that the robustness of our models are not due to degrading the functionality of PGD attacks, here we verify that our models indeed have zero accuracy when the adversary is allowed to make huge perturbations. In Table 9 by performing an unbounded PGD attack on a sample logit-squeezed model for the CIFAR-10 (β = 10, σ = 30, and k = 160k), we verify that our attack is not benefiting from obfuscating the gradients. Table 9: The effect of unbounded on the accuracy. The decline in the accuracy as a sanity check shows that the sample model is at least not completely breaking the PGD attack and is not obfuscating the gradients. We attack our sample hardened CIFAR-10 model (β = 10, Gaussian augmentation σ = 30 model, and k = 160k iterations), by performing many random restarts. Random restarts can potentially increase the strength of the PGD attack that has a random initialization. As shown in Table 10, increasing the number of random restarts does not significantly degrade the robustness. It should be noted that attacking with more iterations and more random restarts hurts adversarially trained models as well (see the leaderboard in Madry's Cifar10 Github Repository). Table 10: The effect of the number of random restarts while generating the adversarial examples on the accuracy of a model trained with the logit squeezing. It shows that the accuracy plateaus at 9 random restarts. Another way to increase the strength of an adversary is by increasing the number of PGD steps for the attack. We notice that increasing the number of steps does not greatly affect the robustness of our sample logit-squeezed model (See Table 11). Table 11: The effect of the number of steps of the white-box PGD attack on the CW loss (worst case based on TAB3) for the model trained in 160k steps with logit squeezing parameters β = 10 and σ = 30 on CIFAR-10 dataset. The model remains resistant against ∞ attacks with = 8 Figure 9: The cross-entropy landscape of the first eight images of the validation set for the model trained for 160k iteration and hyper-parameters β = 10 and σ = 30. To plot the loss landscape we take walks in one random direction r 1 ∼ Rademacher(0.5) and the adversarial direction a 2 = sign(∇ x xent) where xent is the cross-entropy loss. We plot the cross-entropy (i.e. xent) loss at different points x = x i + 1 · r 1 + 2 · a 2. Where x i is the clean image and −10 ≤ 1, 2 ≤ 10. As it can be seen moving along the adversarial direction changes the loss value a lot and moving along the random direction does not make any significant major changes. Similar to Figure 5, we plot the classification loss landscape surrounding the input images for the first eight images of the validation set. Unlike in Figure 5 which we changed the clean image along two random directions, in Figure 9, we wander around the clean image by moving in the adversarial direction and one random direction. From Figure 9 we observe that the true direction that the classification loss changes is along the adversarial direction which illustrates that the logit-squeezed model is not masking the gradients. 2) with random noise, and logit squeezing (β = 0.5) with random noise. In all cases, the noise is Gaussian with σ = 0.5. Interestingly, the combination of Gaussian noise and label smoothing, similar to the combination of Gaussian noise and logit squeezing, deactivates roughly 400 neurons. This is similar to adversarial training. In some sense it seems that all three methods are causing the "effective" dimensionality of the deep feature representation layer to shrink. FIG0: For MNIST, we plot the cumulative sum of activation magnitudes for all neurons in logit layer of a network produced by natural training, adversarial training, natural training with random noise, label smoothing (LS = 0.2) with random noise, and logit squeezing (β = 0.5) with random noise. In all cases, the noise is Gaussian with σ = 0.5.

We can identify prototypical and outlier examples in machine learning that are quantifiably very different, and make use of them to improve many aspects of neural networks.