Question: Represent the scientific article to find a short TLDR

Question: Adversarial training is one of the main defenses against adversarial attacks. In this paper, we provide the first rigorous study on diagnosing elements of large-scale adversarial training on ImageNet, which reveals two intriguing properties. First, we study the role of normalization. Batch normalization (BN) is a crucial element for achieving state-of-the-art performance on many vision tasks, but we show it may prevent networks from obtaining strong robustness in adversarial training. One unexpected observation is that, for models trained with BN, simply removing clean images from training data largely boosts adversarial robustness, i.e., 18.3%. We relate this phenomenon to the hypothesis that clean images and adversarial images are drawn from two different domains. This two-domain hypothesis may explain the issue of BN when training with a mixture of clean and adversarial images, as estimating normalization statistics of this mixture distribution is challenging. Guided by this two-domain hypothesis, we show disentangling the mixture distribution for normalization, i.e., applying separate BNs to clean and adversarial images for statistics estimation, achieves much stronger robustness. Additionally, we find that enforcing BNs to behave consistently at training and testing can further enhance robustness. Second, we study the role of network capacity. We find our so-called "deep" networks are still shallow for the task of adversarial learning. Unlike traditional classification tasks where accuracy is only marginally improved by adding more layers to "deep" networks (e.g., ResNet-152), adversarial training exhibits a much stronger demand on deeper networks to achieve higher adversarial robustness. This robustness improvement can be observed substantially and consistently even by pushing the network capacity to an unprecedented scale, i.e., ResNet-638. Adversarial attacks can mislead neural networks to make wrong predictions by adding human imperceptible perturbations to input data. Adversarial training is shown to be an effective method to defend against such attacks, which trains neural networks on adversarial images that are generated on-the-fly during training. Later works further improve robustness of adversarially trained models by mitigating gradient masking (Tramèr et al., 2018), imposing logits pairing , denoising at feature space (b), etc. However, these works mainly focus on justifying the effectiveness of proposed strategies and apply inconsistent pipelines for adversarial training, which leaves revealing important elements for training robust models still a missing piece in current adversarial research. In this paper, we provide the first rigorous diagnosis of different adversarial learning strategies, under a unified training and testing framework, on the large-scale ImageNet dataset . We discover two intriguing properties of adversarial training, which are essential for training models with stronger robustness. First, though Batch Normalization (BN) is known as a crucial component for achieving state-of-the-arts on many vision tasks, it may become a major obstacle for securing robustness against strong attacks in the context of adversarial training. By training such networks adversarially with different strategies, e.g., imposing logits pairing , we observe an unexpected phenomenon -removing clean images from training data is the most effective way for boosting model robustness. We relate this phenomenon to the conjecture that clean images and adversarial images are drawn from two different domains. This two-domain hypothesis may explain the limitation of BN when training with a mixture of clean and adversarial images, as estimating normalization statistics on this mixture distribution is challenging. We further show that adversarial training without removing clean images can also obtain strong robustness, if the mixture distribution is well disentangled at BN by constructing different mini-batches for clean images and adversarial images to estimate normalization statistics, i.e., one set of BNs exclusively for adversarial images and another set of BNs exclusively for clean images. An alternative solution to avoiding mixture distribution for normalization is to simply replace all BNs with batch-unrelated normalization layers, e.g., group normalization , where normalization statistics are estimated on each image independently. These facts indicate that model robustness is highly related to normalization in adversarial training. Furthermore, additional performance gain is observed via enforcing consistent behavior of BN during training and testing. Second, we find that our so-called "deep" networks (e.g., are still shallow for the task of adversarial learning, and simply going deeper can effectively boost model robustness. Experiments show that directly adding more layers to "deep" networks only marginally improves accuracy for traditional image classification tasks. In contrast, substantial and consistent robustness improvement is witnessed even by pushing the network capacity to an unprecedented scale, i.e., ResNet-638. This phenomenon suggests that larger networks are encouraged for the task of adversarial learning, as the learning target, i.e., adversarial images, is a more complex distribution than clean images to fit. In summary, our paper reveals two intriguing properties of adversarial training: properly handling normalization is essential for obtaining models with strong robustness; and our so-called "deep" networks are still shallow for the task of adversarial learning. We hope these findings can benefit future research on understanding adversarial training and improving adversarial robustness. Adversarial training. Adversarial training constitutes the current foundation of state-of-the-arts for defending against adversarial attacks. It is first developed in where both clean images and adversarial images are used for training. propose to improve robustness further by encouraging the logits from the pairs of clean images and adversarial counterparts to be similar. Instead of using both clean and adversarial images for training, formulate adversarial training as a min-max optimization and train models exclusively on adversarial images. Subsequent works are then proposed to further improve the model robustness (b; b; a; ; ; b; ;) or accelerate the adversarial training process (; a; . However, as these works mainly focus on demonstrating the effectiveness of their proposed mechanisms, a fair and detailed diagnosis of large-scale adversarial training strategies remains as a missing piece. In this work, we provide the first detailed diagnosis which reveals two intriguing properties of training adversarial defenders at scale. Normalization Layers. Normalization is an effective technique to accelerate the training of deep networks. Different methods are proposed to exploit batch-wise (e.g., BN ), layer-wise (e.g., layer normalization ) or channel-wise (e.g., instance normalization and group normalization ) information for estimating normalization statistics. Different from traditional vision tasks where BN usually yields stronger performance than other normalization methods, we show that BN may become a major obstacle for achieving strong robustness in the context of adversarial training, and properly handling normalization is an essential factor to improving adversarial robustness. As inconsistent adversarial training pipelines were applied in previous works (; b), it is hard to identify which elements are important for obtaining robust models. To this end, we provide a unified framework to train and to evaluate different models, for the sake of fair comparison. Training Parameters. We use the publicly available adversarial training pipeline 1 to train all models with different strategies on ImageNet. We select ResNet-152 as the baseline network, and apply projected gradient descent (PGD) as the adversarial attacker to generate adversarial examples during training. The hyper-parameters of the PGD attacker are: maximum perturbation of each pixel = 16, attack step size α = 1, number of attack iterations N = 30, and the targeted class is selected uniformly at random over the 1000 ImageNet categories. We initialize the adversarial image by the clean counterpart with probability = 0.2, or randomly within the allowed cube with probability = 0.8. All models are trained for a total of 110 epochs, and we decrease the learning rate by 10× at the 35-th, 70-th, and 95-th epoch. Evaluation. For performance evaluation, we mainly study adversarial robustness (rather than clean image accuracy) in this paper. Specifically, we follow the setting in and Xie et al. (2019b), where the targeted PGD attacker is chosen as the white-box attacker to evaluate robustness. The targeted class is selected uniformly at random. We constrain the maximum perturbation of each pixel = 16, set the attack step size α = 1, and measure the robustness by defending against PGD attacker of 2000 attack iterations (i.e., PGD-2000). As in and Xie et al. (2019b), we always initialize the adversarial perturbation from a random point within the allowed -cube. We apply these training and evaluation settings by default for all experiments, unless otherwise stated. In this part, we first elaborate on the effectiveness of different adversarial training strategies on model robustness. Adversarial training can be dated back to , where they mix clean images and the corresponding adversarial counterparts into each mini-batch for training. We choose this strategy as our starting point, and the corresponding loss function is: where J(·) is the loss function, θ is the network parameter, y is the ground-truth, and training pairs {x clean, x adv} are comprised of clean images and their adversarial counterparts, respectively. The parameter α balances the relative importance between clean image loss and adversarial image loss. We set α = 0.5 following. With our adversarial training framework, this model can achieve 20.9% accuracy against PGD-2000 attacker. Besides this baseline, we also study the effectiveness of two recently proposed adversarial training strategies ), and provide the as follows. Ratio of clean images. Different from the canonical form in , apply the min-max formulation for adversarial training where no clean images are used. We note this min-max type optimization can be dated as early as. We hereby investigate the relationship between model robustness and the ratio of clean images used for training. Specifically, for each training mini-batch, we keep adversarial images unchanged, but removing their clean counterparts by 20%, 40%, 60%, 80% and 100%. We report the in Figure 1. Interestingly, removing a portion of clean images from training data can significantly improve model robustness, and the strongest robustness can be obtained by completely removing clean images from the training set, i.e., it achieves an accuracy of 39.2% against PGD-2000 attacker, outperforming the baseline model by a large margin of 18.3%. Adversarial logits pairing. For performance comparison, we also explore the effectiveness of an alternative training strategy, adversarial logits pairing (ALP) . Compared with the canonical form in , ALP imposes an additional loss to encourage the logits from the pairs of clean images and adversarial counterparts to be similar. As shown in Figure 2, our re-implemented ALP obtains an accuracy of 23.0% against PGD-2000 attacker 2, which outperforms the baseline model by 2.1%. Compared with the strategy of removing clean images, this improvement is much smaller. Discussion. Given the above, we conclude that training exclusively on adversarial images is the most effective strategy for boosting model robustness. For example, by defending against PGD-2000 attacker, the baseline strategy in (referred to as 100% adv + 100% clean) obtains an accuracy of 20.9%. Adding an loss of logits pairing (referred to as 100% adv + 100% clean, ALP) slightly improves the performance by 2.1%, while completely removing clean images b ) (referred to as 100% adv + 0% clean) boosts the accuracy by 18.3%. We further plot a comprehensive evaluation curve of these three training strategies in Figure 2, by varying the number of PGD attack iteration from 10 to 2000. Surprisingly, only 100% adv + 0% clean can ensure model robustness against strong attacks, i.e., performance becomes asymptotic when allowing PGD attacker to perform more attack iterations. Training strategies which involve clean images for training are suspicious to in worse robustness, if PGD attackers are allowed to perform more attack iterations. In the next section, we will study how to make these training strategies, i.e., 100% adv + 100% clean and 100% adv + 100% clean, ALP to secure their robustness against strong attacks. Two-domain hypothesis. Compared to feature maps of clean images, Xie et al. (2019b) show that feature maps of their adversarial counterparts tend to be more noisy. Meanwhile, several works (Li Figure 3: Disentangling the mixture distribution for normalization secures model robustness. Unlike the blue curves in Figure 2, these new curves become asymptotic when evaluating against attackers with more iterations, which indicate that the networks using MBN adv can behave robustly against PGD attackers with different attack iterations, even if clean images are used for training. & ; ; ; ; demonstrate it is possible to build classifiers to separate adversarial images from clean images. These studies suggest that clean images and adversarial images are drawn from two different domains 3 . This two-domain hypothesis may provide an explanation to the unexpected observation (see Sec. 4.1) and we ask -why simply removing clean images from training data can largely boost adversarial robustness? As a crucial element for achieving state-of-the-arts on various vision tasks, BN is widely adopted in many network architectures, e.g., Inception, ResNet and DenseNet . The normalization statistics of BN are estimated across different images. However, exploiting batch-wise statistics is a challenging task if input images are drawn from different domains and therefore networks fail to learn a unified representation on this mixture distribution. Given our two-domain hypothesis, when training with both clean and adversarial images, the usage of BN can be the key issue for ing in weak adversarial robustness in Figure 2. Based on the analysis above, an intuitive solution arise: accurately estimating normalization statistics should enable models to train robustly even if clean images and adversarial images are mixed at each training mini-batch. To this end, we explore two ways, where the mixture distribution is disentangled at normalization layers, for validating this argument: maintaining separate BNs for clean/adversarial images; or replacing BNs with batch-unrelated normalization layers. Training with Mixture BN. Current network architectures estimate BN statistics using the mixed features from both clean and adversarial images, which leads to weak model robustness as shown in Figure 2. Xie et al. (2019a) propose that properly decoupling the normalization statistics for adversarial training can effectively boost image recognition. Here, to study model robustness, we apply Mixture BN (MBN) (a), which disentangles the mixed distribution via constructing different mini-batches for clean and adversarial images for accurate BN statistics estimation (illustrated in Figure 4), i.e., one set of BNs exclusively for adversarial images (referred to as MBN adv), and another set of BNs exclusively for clean images (referred to as MBN clean). We do not change the structure of other layers. We verify the effectiveness of this new architecture with two (previously less robust) training strategies, i.e., 100% adv + 100% clean and 100% adv + 100% clean, ALP. At inference time, whether an image is adversarial or clean is unknown. We thereby measure the performance of networks by applying either MBN adv or MBN clean separately. The are shown in Table 1. We find the performance is strongly related to how BN is trained: when using MBN clean, the trained network achieves nearly the same clean image accuracy as the whole network trained exclusively on clean images; when using MBN adv, the trained network achieves nearly the same adversarial robustness as the whole network trained exclusively on adversarial images. Other factors, like whether ALP is applied for training, only cause subtle differences in performance. We further plot an extensive robustness evaluation curve of different training strategies in Figure 3. Unlike Figure 2, we observe that networks using MBN adv now can secure their robustness against strong attacks, e.g., the robustness is asymptotic when increasing attack iterations from 500 to 2000. The in Table 1 suggest that BN statistics characterize different model performance. For a better understanding, we randomly sample 20 channels in a residual block and plot the corresponding running statistics of MBN clean and MBN adv in Figure 5. We observe that clean images and adversarial images induce significantly different running statistics, though these images share the same set of convolutional filters for feature extraction. This observation further supports that clean images and adversarial images come from two different domains; and current networks fail to learn a unified representation on these two domains. Interestingly, we also find that adversarial images lead to larger running mean and variance than clean images. This phenomenon is also consistent with the observation that adversarial images produce noisy-patterns/outliers at the feature space (b). As a side note, this MBN structure is also used as a practical trick for training better generative adversarial networks (GAN). suggest to construct each mini-batch with only real or generated images when training discriminators, as generated images and real images belong to different domains at an early training stage. However, unlike our situation where BN statistics estimated on different domains remain divergent after training, a successful training of GAN, i.e., able to generate natural images with high quality, usually learns a unified set of BN statistics on real and generated images. Training with batch-unrelated normalization layers. Instead of applying MBN structure to disentangle the mixture distribution, we can also train networks with batch-unrelated normalization layers, which avoids exploiting the batch dimension to calculate statistics, for the same purpose. We choose Group Normalization (GN) for this experiment, as GN can reach a comparable performance to BN on various vision tasks (Table 2 : Enforcing a consistent behavior of BN at the training stage and the testing stage significantly boosts adversarial robustness. * denotes that running statistics is used at the last 10 training epochs. channels into groups and computes the normalization statistics within each group. By replacing all BNs with GNs, the mixture training strategy 100% adv + 100% clean now can ensure robustness against strong attacks, i.e., the model trained with GN achieves 39.5% accuracy against PGD-500, and increasing attack iterations to 2000 only cause a marginal performance drop by 0.5% (39.0% accuracy against PGD-2000). Exploring other batch-unrelated normalization in adversarial training remains as future work. Exceptional cases. There are some situations where models directly trained with BN can also ensure their robustness against strong attacks, even if clean images are included for adversarial training. Our experiments show constraining the maximum perturbation of each pixel to be a smaller value, e.g., = 8, is one of these exceptional cases. and also show that adversarial training with clean images can secure robustness on small datasets, i.e., MNIST, CIFAR-10 and Tiny ImageNet. Intuitively, generating adversarial images on these much simpler datasets or under a smaller perturbation constraint induces a smaller gap between these two domains, therefore making it easier for networks to learn a unified representation on clean and adversarial images. Nonetheless, in this paper, we stick to the standard protocol in and Xie et al. (2019b) where adversarial robustness is evaluated on ImageNet with the perturbation constraint = 16. Inconsistent behavior of BN. As the concept of "batch" is not legitimate at inference time, BN behaves differently at training and testing : during training, the mean and variance are computed on each mini-batch, referred to as batch statistics; during testing, there is no actual normalization performed -BN uses the mean and variance pre-computed on the training set (often by running average) to normalize data, referred to as running statistics. For traditional classification tasks, batch statistics usually converge to running statistics by the end of training, thus (practically) making the impact of this inconsistent behavior negligible. Nonetheless, this empirical assumption may not hold in the context of adversarial training. We check this statistics matching of models trained with the strategy 100% adv + 0% clean, where the robustness against strong attacks is secured. We randomly sample 20 channels in a residual block, and plot the batch statistics computed on two randomly sampled mini-batches, together with the pre-computed running statistics. In Figure 6, interestingly, we observe that batch mean is almost equivalent to running mean, while batch variance does not converge to running variance yet on certain channels. Given this fact, we then study if this inconsistent behavior of BN affects model robustness in adversarial training. A heuristic approach. Instead of developing a new training strategy to make batch statistics converge to running statistics by the end of training, we explore a more heuristic solution: applying pre-computed running statistics for model training during the last 10 epochs. We report the performance comparison in Table 2. By enabling BNs to behave consistently at training and testing, this approach can further boost the model robustness by 3.0% with the training strategy 100% adv + 0% clean. We also successfully validate the generality of this approach on other two robust training strategies. More specifically, it can improve the model robustness under the training strategies MBN adv, 100% adv + 100% clean and MBN adv, 100% adv + 100% clean, ALP by 1.6% and 2.8%, respectively. These suggest that model robustness can be benefited from a consistent behavior of BN at training and testing. Moreover, we note this approach does not incur any additional training budgets. On the importance of training convolutional filters adversarially. In Section 4.2, we study the performance of models where the mixture distribution is disentangled for normalization -by applying either MBN clean or MBN adv, the trained models achieve strong performance on either clean images or adversarial images. This suggests that clean and adversarial images share the same convolutional filters to effectively extract features. We further explore whether the filters learned exclusively on adversarial images can extract features effectively on clean images, and vice versa. We first take a model trained with the strategy 100% adv + 0% clean, and then finetune BNs using only clean images for a few epochs. Interestingly, we find the accuracy on clean images can be significantly boosted from 62.3% to 73%, which is only 5.9% worse than the standard training setting, i.e., 78.9%. These indicates that convolutional filters learned solely on adversarial images can also be effectively applied to clean images. However, we find the opposite direction does not work -convolutional filters learned on clean images cannot extract features robustly on adversarial images (e.g., 0% accuracy against PGD-2000 after finetuning BNs with adversarial images). This phenomenon indicates the importance of training convolutional filters adversarially, as such learned filters can also extract features from clean images effectively. The findings here also are related to the discussion of robust/non-robustness features in. Readers with interests are recommended to refer to this concurrent work for more details. Limitation of adversarial training. We note our adversarially trained models exhibit a performance tradeoff between clean accuracy and robustness -the training strategies that achieve strong model robustness usually in relatively low accuracy on clean images. For example, 100% adv + 0% clean, MBN adv, 100% adv + 100% clean and MBN adv, 100% adv + 100% clean, ALP only report 62.3%, 64.4% and 65.9% of clean image accuracy. By replacing BNs with GNs, 100% adv + 100% clean achieves much better clean image accuracy, i.e., 67.5%, as well maintaining strong robustness. We note that this tradeoff is also observed in the prior work. show it is possible to make adversarially trained models to exhibit a better tradeoff between clean accuracy and robustness. Future attentions are deserved on this direction. As discussed in Section 4.2, current networks are not capable of learning a unified representation on clean and adversarial images. It may suggest that the "deep" network we used, i.e., ResNet-152, still underfits the complex distribution of adversarial images, which motivates us to apply larger networks for adversarial training. We simply instantiate the concept of larger networks by going deeper, i.e., adding more residual blocks. For traditional image classification tasks, the benefits brought by adding more layers to "deep" networks is diminishing, e.g., the blue curve in Figure 7 shows that the improvement of clean image accuracy becomes saturated once the network depth goes beyond ResNet-200. For a better illustration, we train deeper models exclusively on adversarial images and observe a possible underfitting phenomenon as shown in Figure 7. In particular, we apply the heuristic policy in Section 4.3 to mitigate the possible effects brought by BN. We observe that adversarial learning task exhibits a strong "thirst" on deeper networks to obtain stronger robustness. For example, increasing depth from ResNet-152 to ResNet-338 significantly improves the model robustness by 2.4%, while the corresponding improvement in the "clean" training setting (referred to as 0% adv + 100% clean) is only 0.5%. Moreover, this observation still holds even by pushing the network capacity to an unprecedented scale, i.e., ResNet-638. These indicate that our so-called "deep" networks (e.g., ResNet-152) are still shallow for the task of adversarial learning, and larger networks should be used for fitting this complex distribution. Besides our findings on network depth, show increase network width also substantially improve network robustness. These empirical observations also corroborate with the recent theoretical studies which argues that robust adversarial learning needs much more complex classifiers. Besides adversarial robustness, we also observe a consistent performance gain on clean image accuracy by increasing network depth (as shown in Table 7). Our deepest network, ResNet-638, achieves an accuracy of 68.7% on clean images, outperforming the relatively shallow network ResNet-152 by 6.1%. In this paper, we reveal two intriguing properties of adversarial training at scale: conducting normalization in the right manner is essential for training robust models on large-scale datasets like ImageNet; and our so-called "deep" networks are still shallow for the task of adversarial learning. Our discoveries may also be inherently related to our two-domain hypothesis -clean images and adversarial images are drawn from different distributions. We hope these findings can facilitate fellow researchers for better understanding of adversarial training as well as further improvement of adversarial robustness. In the main paper, we note that our reproduced ALP significantly outperforms the reported in , as well in an independent study. The main differences between our version and the original ALP implementation lie in parameter settings, and are detailed as follows: • learning rate decay: the original ALP decays the learning rate every two epochs at an exponential rate of 0.94, while ours decays the learning rate by 10× at the 35-th, 70-th and 95-th epoch. To ensure these two policies reach similar learning rates by the end of training, the total number of training epochs of the exponential decay setting and the step-wise decay setting are set as 220 and 110 respectively. • initial learning rate: the original ALP sets the initial learning rate as 0.045 whereas we set it as 0.1 in our implementation. • training optimizer: the original ALP uses RMSProp as the optimizer while we use Momentum SGD (M-SGD). • PGD initialization during training: the original ALP initializes the adversarial perturbation from a random point within the allowed cube; while we initialize the adversarial image by its clean counterpart with probability = 0.2, or randomly within the allowed the cube with probability = 0.8. Table 3: The of ALP re-implementations under different parameter settings. We show that applying stronger attackers for training, e.g., change from PGD-10 to PGD-30, is the most important factor for achieving strong robustness. Other parameters, like optimizer, do not lead to significant robustness changes. By following the parameter settings listed in the ALP paper 4, we can train a ResNet-101 with an accuracy of 38.1% against PGD-10. The ResNet-101 performance reported in the ALP paper is 30.2% accuracy against an attack suite 5. This ∼8% performance gap is possibly due to different attacker settings in evaluation. However, by evaluating this model against PGD-2000, we are able to obtain a similar that reported in, i.e., reports ALP obtains 0% accuracy, and in our implementation the accuracy is 2.1%. Given these different settings, we change them one by one to train corresponding models adversarially. The are summarized in Table 3. Surprisingly, we find the most important factor for the performance gap between original ALP paper and ours is the attacker strength used for trainingby changing the attacker from PGD-10 to PGD-30 for training, the robustness against PGD-2000 can be increased by 19.7%. Other parameters, like network backbones or the GPU number, do not lead to significant performance changes. In this section, we explore the impact of different training parameters on model performance. As suggested in Table 3, the number of attack iteration used for training is an important factor for model robustness. We hereby provide a detailed diagnosis of model performance trained with PGD-{5, 10, 20} 6 for different training strategies. We report the performance in Table 4, and observe that decreasing the number of PGD attack iteration used for training usually leads to weaker robustness. Nonetheless, we note the amount of this performance change is strongly related to training strategies. For strategies that cannot lead to models with strong robustness, i.e., 100% adv + 100% clean and 100% adv + 100% clean, ALP, this robustness degradation is extremely severe (which is similar to the observation in Table 3). For example, by training with PGD-5, these two strategies obtains nearly no robustness, i.e., ∼0% accuracy against PGD-2000. While for strategies that can secure model robustness against strong attacks, changing from PGD-30 to PGD-5 for training only lead to a marginal robustness drop. Table 4: Robustness evaluation of models adversarially trained with PGD-{30, 20, 10, 5} attackers. We observe that decreasing the number of PGD attack iteration for training usually leads to weaker robustness, while the amount of degraded robustness is strongly related to training strategies. In Section 4.3 (of the main paper), we study the effectiveness of applying running statistics in training. We hereby test this heuristic policy under more different settings. Specifically, we consider 3 strategies, each trained with 4 different attackers (i.e., PGD-{5, 10, 20, 30}), which in 12 different settings. We report the in Table 5. We observe this heuristic policy can boost robustness on all settings, which further supports the importance of enforcing BN to behave consistently at training and testing. In the main paper, our study is driven by improving adversarial robustness (measured by the accuracy against PGD-2000), while leaving the performance on clean images ignored. For the completeness of performance evaluation, we list the clean image performance of these adversarially trained models in Table 7. Moreover, to facilitate performance comparison in future works, we list the corresponding accuracy against PGD-{10, 20, 100, 500} in this

Represent this text to find a much much longer article about it with say hundreds-thousands of words

The first rigor diagnose of large-scale adversarial training on ImageNet

We find that deep networks which generalize poorly are more reliant on single directions than those that generalize well, and evaluate the impact of dropout and batch normalization, as well as class selectivity on single direction reliance.


Question: Represent the text to find a very short summary (1-2 sentences max)

Question: The gradient of a deep neural network (DNN) w.r.t. the input provides information that can be used to explain the output prediction in terms of the input features and has been widely studied to assist in interpreting DNNs. In a linear model (i.e., $g(x)=wx+b$), the gradient corresponds solely to the weights $w$. Such a model can reasonably locally linearly approximate a smooth nonlinear DNN, and hence the weights of this local model are the gradient. The other part, however, of a local linear model, i.e., the bias $b$, is usually overlooked in attribution methods since it is not part of the gradient. In this paper, we observe that since the bias in a DNN also has a non-negligible contribution to the correctness of predictions, it can also play a significant role in understanding DNN behaviors. In particular, we study how to attribute a DNN's bias to its input features. We propose a backpropagation-type algorithm ``bias back-propagation (BBp)'' that starts at the output layer and iteratively attributes the bias of each layer to its input nodes as well as combining the ing bias term of the previous layer. This process stops at the input layer, where summing up the attributions over all the input features exactly recovers $b$. Together with the backpropagation of the gradient generating $w$, we can fully recover the locally linear model $g(x)=wx+b$. Hence, the attribution of the DNN outputs to its inputs is decomposed into two parts, the gradient $w$ and the bias attribution, providing separate and complementary explanations. We study several possible attribution methods applied to the bias of each layer in BBp. In experiments, we show that BBp can generate complementary and highly interpretable explanations of DNNs in addition to gradient-based attributions. Deep neural networks (DNNs) have produced good for many challenging problems in computer vision, natural language processing, and speech processing. Deep learning models, however, are usually designed using fairly high-level architectural decisions, leading to a final model that is often seen as a difficult to interpret black box. DNNs are a highly expressive trainable class of non-linear functions, utilizing multi-layer architectures and a rich set of possible hidden non-linearities, making interpretation by a human difficult. This restricts the reliability and usability of DNNs especially in mission-critical applications where a good understanding of the model's behavior is necessary. The gradient is a useful starting point for understanding and generating explanations for the behavior of a complex DNN. Having the same dimension as the input data, the gradient can reflect the contribution to the DNN output of each input dimension. Not only does the gradient yield attribution information for every data point, but also it helps us understand other aspects of DNNs, such as the highly celebrated adversarial examples and defense methods against such attacks BID13.When a model is linear, the gradient recovers the weight vector. Since a linear model locally approximates any sufficiently smooth non-linear model, the gradient can also be seen as the weight vector of that local linear model for a given DNN at a given data point. For a piecewise linear DNN (e.g., a DNN with activation functions such as ReLU, LeakyReLU, PReLU, and hard tanh) the gradient is exactly the weights of the local linear model 1.Although the gradient of a DNN has been shown to be helpful in understanding the behavior of a DNN, the other part of the locally linear model, i.e., the bias term, to the best of our knowledge, has not been studied explicitly and is often overlooked. If only considering one linear model within a small region, the bias, as a scalar, seems to contain less information than the weight vector. However, this scalar is the of complicated processing of bias terms over every neuron and every layer based on the activations, the non-linearity functions, as well as the weight matrices of the network. Uncovering the bias's nature could potentially reveal a rich vein of attribution information complementary to the gradient. For classification tasks, it can be the case that the gradient part of the linear model contributes to only a negligible portion of the target label's output probability (or even a negative logit value), and only with a large bias term does the target label's probability becomes larger than that of other labels to in the correct prediction (see Sec 5). In our empirical experiments TAB0, using only the bias term of the local linear models achieves 30-40% of the performance of the complete DNN, thus indicating that the bias term indeed plays a substantial role in the mechanisms of a DNN.In this paper, we unveil the information embedded in the bias term by developing a general bias attribution framework that distributes the bias scalar to every dimension of the input data. We propose a backpropagation-type algorithm called "bias backpropagation (BBp)" to send and compute the bias attribution from the output and higher-layer nodes to lower-layer nodes and eventually to the input features, in a layer-by-layer manner. Specifically, BBp utilizes a recursive rule to assign the bias attribution on each node of layer to all the nodes on layer − 1, while the bias attribution on each node of layer − 1 is composed of the attribution sent from the layer below and the bias term incurred in layer − 1. The sum of the attributions over all input dimensions produced by BBp exactly recovers the bias term in the local linear model representation of the DNN at the given input point. In experiments, we visualize the bias attribution as images on a DNN trained for image classification. We show that bias attribution can highlight essential features that are complementary from what the gradient-alone attribution methods favor. Attribution methods for deep models is an important modern research in machine learning since it is important to complement the good empirical performance of DNNs with explanations for how, why, and in what manner do such complicated models make their decisions. Ideally, such methods would render DNNs to be glass boxes rather than black boxes. To this end, a number of strategies have been investigated. BID10 visualized behaviors of convolutional networks by investigating the gradients of the predicted class output with respect to the input features. Deconvolution BID15 and guided backpropagation BID11 ) modify gradients with additional constraints. BID5 extended to higher order gradient information by calculating the Taylor expansion, and BID0 study the Taylor expansion approach on DNNs with local renormalization layers. BID8 proposed DeepLift, which separated the positive attribution and negative attribution, and featured customer designed attribution scores. BID12 declared two axioms an attribution method needs to satisfy. It further developed an integrated gradient method that accumulates gradients on a straightline path from a base input to a real data point and uses the aggregated gradients to measure the importance of input features. Class Activation Mapping (CAM) BID16 localizes the attribution based on the activation of convolution filters, and can only be applied to a fully convolutional network. Grad-CAM BID7 relaxes the all-convolution constraints of CAM by incorporating the gradient information from the non-convolutional layers. All the work mentioned above utilizes information encoded in the gradients in some form or another, but none of them explicitly investigates the importance of the bias terms, which is the focus of this paper. Some of them, e.g. BID8 and BID12, consider the overall activation of neurons in their attribution methods, so the bias terms are implicitly taken into account, but are not independently studied. Moreover, some of the previous work (e.g. CAM) focus on the attribution for specific network architectures such as convolutional networks, while our approach generally applies to any piece-wise linear DNN, convolutional or otherwise. We can write the output f (x) of any feed-forward deep neural network in the following form: DISPLAYFORM0 Where W i and b i are the weight matrix and bias term for layer i, ψ i is the corresponding activation function, x ∈ X is an input data point of d in dimensions, f (x) is the network's output prediction of d out dimensions, and each hidden layer i has d i nodes. In this paper, we rule out the last softmax layer from the network structure; for example, the output f (x) may refer to logits (which are the inputs to a softmax to compute probabilities) if the DNN is trained for classification tasks. The above formalization of DNN generalizes many widely used DNN architectures. Clearly, Eq. can represent a fully-connected network of m layers. Moreover, the convolution operation is essentially a matrix multiplication, where every row of the matrix corresponds to applying a filter from convolution on a certain part of the input, and therefore the ing weight matrix has tied parameters and is very sparse, and typically has a very large (compared to the input size) number of rows. The average-pooling is essentially a linear operation and therefore is representable as a matrix multiplication, and max-pooling can be treated as an activation function. Batchnorm BID3 ) is a linear operation and can be combined into the weight matrix. Finally, we can represent a residual network BID2 block by appending an identity matrix at the bottom of a weight matrix so that we can keep the input values, and then add the kept input values later through another matrix operation. In this paper, we will focus on DNNs with piecewise linear activation functions, which cover most of the successfully used neural networks in a variety of application domains. Some widely used piecewise linear activation functions include the ReLU, leaky ReLU, PReLU, and the hard tanh functions. A general form of a piecewise linear activation function applied to a real value z is as follows: DISPLAYFORM0 In the above, there are h linear pieces, and these correspond to h predefined intervals on the real axis. We define the activation pattern φ(z) of z as the index of the interval containing z, which can be any integer from 0 to h − 1. Both ψ(z) and φ(z) extend to element-wise operators when applied to vectors or high dimensional tensors. As long as the activation function is piecewise linear, the DNN is a piecewise linear function and is equivalent to a linear model at each input point x. Specifically, each piece of the DNN (associated with an input point x) is a linear model: DISPLAYFORM1 This holds true for all the possible input points x on the linear piece of DNN. We will give a more general later in Lemma 1. Note W DISPLAYFORM2 where x i is the activation of layer i (x 1 is the input data) and b x i is an x i -dependent bias vector vector. In the extreme case, no two input training data points share the same linear model in Eq.. Note in this case, the DNN can still be represented as a piecewise linear model on a set of input data points, and each local linear model is only applied to one data point. DISPLAYFORM3 For instance, if ReLU ψ ReLU (z) = max(0, z) is used as the activation function ψ(·) at every layer i, we have an activation pattern φ(DISPLAYFORM4 indicates that ReLU sets the output node p to 0 or otherwise preserves the node value. Therefore, at layer i, W to be 0 while other elements to be b i .We can apply the above process to deeper layers as well, eliminating all the ReLU functions to produce an x-specific local linear model representing one piece of the DNN, as shown in Eq.. Since the model is linear, the gradient DISPLAYFORM5 ∂x is the weight vector of the linear model. Also, given all the weights of the DNN, each linear region, and the associated linear model can be uniquely determined by the ReLU patterns {φ(x i)} m i=2, which are m binary vectors. Given a specific input point x, the attribution of each dimension f (x)[j] of the DNN output (e.g., the logit for class j) to the input features aims to assign a portion of f (x)[j] to each of the input features i, and all the portions assigned to all the input features should sum up to f (x) [j]. For simplicity, in the rest of this paper, we rename f (x) to be f (x)[j], which does not lose any generality since the same attribution method can be applied for any output dimension j. According to Eq., f (x) as a linear model on x can be decomposed into two parts, the linear transformation DISPLAYFORM0 ∂x and the bias term bx. The attribution of the first part is straightforward because we can directly assign each dimension of the gradient DISPLAYFORM1 ∂x to the associated input feature, and we can generate the gradient by using the standard backpropagation algorithm. The gradient-based attribution methods have been widely studied in previous work (see Section 2). However, the attribution of the second part, i.e., the bias b, is arguably a more challenging problem since it is not obvious how to assign a portion of b to each input feature since b is a scalar value rather than a vector that, like the gradient, has the same dimensionality as the input vector. One possible reason for the dearth of bias attribution studies might be that people consider bias, as a scalar, less important relative to the weight vector, containing only minor information about deep model decisions. The final bias scalar bx of every local linear model, however, is the of a complex process (see Eq. FORMULA2), where the bias term on every neuron of a layer gets modified based on the activation function (e.g., for ReLU, a bias term gets dropped if the neuron has a negative value), then propagates to the next layer based on the weight matrix, and contribute to the patterns of activation function in the next layer. As the bias term applied to every neuron can be critical in determining the activation pattern (e.g., changing a neuron output from negative to positive for ReLU), we wish to be able to better understand the behavior of deep models by unveiling and reversing the process of how the final bias term is generated. Moreover, as we show in our empirical studies (see Section 5), we train DNNs both with and without bias for image classification tasks, and the show that the bias plays a significant role in producing accurate predictions. In fact, we find that it is not rare that the main component of a final logit, leading to the final predicted label, comes from the bias term, while the gradient term ∂f (x) ∂x x makes only a minor, or even negative, contribution to the ultimate decision. In such a case, ignoring the bias term can provide misleading input feature attributions. Intuitively, the bias component also changes the geometric shape of the piecewise linear DNNs (see FIG4); this means that it is an essential component of deep models and should also be studied, as we do in this paper. It is a mathematical fact that a piecewise linear DNN is equivalent to a linear model for each input data point. Therefore, the interpretation of the DNN's behavior on the input data should be exclusive to the information embedded in the linear model. However, we often find that the gradient of the DNN, or the weight of the linear model, does not always produce satisfying explanations in practice, and in many cases, it may be due to the overlooked attribution of the bias term that contains the complementary or even key information to make the attribution complete. In this section, we will introduce our method for bias attribution. In particular, the goal is to find a vector β of the same dimension d in as the input data point x such that din p=1 β[p] = b x. However, it is not clear how to directly assign a scalar value b to the d in input dimensions, since there are m layers between the outputs and inputs. In the following, we explore the neural net structure for bias attribution and develop a backpropagation-type algorithm to attribute the bias b layer by layer from the output f (x) to the inputs in a bottom-up manner. Recall x denotes the input nodes of layer ≥ 2, i.e., DISPLAYFORM0 According to the recursive computation shown in Eq., the output f (x) can be represented as a linear model of x shown in the following lemma. Lemma 1. Given x, the output f (x) of a piecewise linear DNN can be written as a linear model of the input x of any layer > 2 (x 1 = x is the raw input) in the following form. DISPLAYFORM1 For each input node x [p] of layer, we aim to compute β [p] as the bias attribution on x [p]. We further require that summing β [p] over all input nodes of layer recovers the bias in Eq. FORMULA10, i.e., DISPLAYFORM2 so the linear model in Eq. FORMULA10 can be represented as the sum of d terms associated with the d input nodes, each composed of a linear transformation part and a bias attribution part, i.e., DISPLAYFORM3 DISPLAYFORM4 We will discuss several options to compute the attribution scores α incurred in layer − 1 (which is applied to all nodes in layer − 1), as shown below: DISPLAYFORM5 It can be easily verified that summing up the attribution β −1 [q] over all the nodes on layer − 1 yields the bias term in Lemma 1, when writing f (x) at x as a linear model of x −1, i.e., DISPLAYFORM6 Hence, the complete attribution of f (x) on the nodes of layer − 1 can be written in the same form as the one shown in Eq. for layer, i.e., f (x) = DISPLAYFORM7. Therefore, we start from the last layer, and recursively apply Eq.- from the last layer to the first layer. This process backpropagates to the lower layers the bias term incurred in each layer and the bias attributions sent from higher layers. Eventually, we can obtain the bias attribution β[p] for each input dimension p. The bias attribution algorithm is detailed in Algorithm 1. DISPLAYFORM8 and {b x} FORMULA0 - FORMULA0 or Eq. FORMULA0; DISPLAYFORM9 // Attribute to the layer input DISPLAYFORM10 // Combine with bias of layer − 1 10 end 11 end 12 return β 1 ∈ R din; In the following, we discuss two possible options to compute the attribution scores in α [p], where α [p, q] measures how much of the bias x [p] should be attributed to x −1 [q]. For both of options, we design α [p] so that the bias attribution on each neuron serves as a compensation for the weight or gradient term to achieve the desired output value, and thus the bias attribution can give complementary explanations to the gradient attribution. We have DISPLAYFORM0 is negative, we may reason that to achieve the target value of x [p], the positive components of the gradient term DISPLAYFORM1 are larger than desirable, so that we need to apply the additional negative bias in order to achieve the desired output x [p]. In other words, the large positive components can be thought as the causal factor leading to the negative bias term, so we attribute more bias to the larger positive components. On the other hand, suppose bx [p] is positive, then the negative components of the gradient term are smaller (or larger in magnitude) than desirable, so the small negative values cause the bias term to be positive, and therefore, we attribute more bias to the smaller negative components. Thus, we have DISPLAYFORM2 where DISPLAYFORM3 DISPLAYFORM4 We use the logistic function to attribute the bias so that the sum over all components recovers the original bias, and T serves as a temperature parameter to control the sharpness of the attribution. With T large, the bias is attributed in a more balanced manner, while with T small, the bias is attributed mostly to a few neurons in layer − 1. Also note that we only consider the non-zero components (indicator 1 e(l−1,p,q)=1 checks whether the component is zero), as the zero-valued components do not offer any contribution to the output value. For example, consider a convolutional layer, the corresponding matrix form is very sparse, and only the non-zero entries are involved in the convolution computation with a filter. The second option adopts a different idea but with a similar philosophy of compensating the attribution of the gradient term. Again, the target is to achieve the value of x [p], and we may assume that to achieve such a value, every component W r=1 1 e(l−1,p,q)=1 (again, we only need to consider the contribution from non-zero components). The offsets of each component to the average target value can be treated as the causal factor of introducing the bias term, where components that are far from the average target get more compensation from the bias, and the components that are close to the target, since they already fulfill their responsibility, receive less compensation from the bias. This produces the following method to compute s [p, q], i.e., DISPLAYFORM5 Note we use the same equations for e(l − 1, p, q) and α l [p, q] as defined in Eq. FORMULA0 - FORMULA0 The above two options are our designs for the α [p, q] function. The attribution function is valid as long as DISPLAYFORM6 While we utilize the logistic function so that all attribution factors are positive, α [p, r] can be negative and still applicable to our BBp framework. We note that there is no single solution to get the optimal attribution function. The proposed options can be applied to any piecewise-linear deep neural networks, and for specific activation function, it is possible to design specialized attribution functions to get still better bias attribution. We first evaluate the importance of bias terms, or in other words, the amount of information encoded in the bias terms by comparing networks trained both with and without bias. In TAB0, we compare on the CIFAR-10,CIFAR-100 BID4 ) and Fashion MNIST BID14 datasets. We trained using the VGG-11 Network of BID9, and we compare the trained with bias, and without bias. Moreover, in the trained with bias case, we derive the linear model of every data point g(x) = wx + b, and compare the performance using only the ing gradient term wx and only the ing bias term b. From the shown in the table, we find that the bias term carries appreciable information and makes unignorable contributions to the correct prediction of the network. We present our bias attribution , using the two options of attribution scores discussed in section 4.2, and compare to the attribution based only on gradient information. We test BBp on STL-10 (BID1 and ImageNet(ILSVRC2012) BID6 and show the in FIG9. For STL-10, we use a 10 layer convolutional network (,maxpool,,maxpool,,maxpool,,maxpool,,dense10, and corresponds to a convolutional layer with 32 channels, kernel size 3 and padding 1), and for ImageNet we use the VGG-11 network of BID9. For both gradient and bias attribution, we select the gradient/bias corresponding to the predicted class (i.e., one row for the final layer weight matrix and one scalar for the final layer bias vector). Note that BBp is a general framework and can work with other choices of gradients and biases for the last layer (e.g. the top predicted class minus the second predicted class).From FIG9, we can observe that the bias attribution can highlight meaningful features in the input data, and in many cases, capture the information that is not covered by the gradient attribution. For example, for the "bird" image of STL-10, the bias attribution shows the major characteristics of a bird such as the body shape, the round head the yellow beak and the claws. Compared to the gradient attribution, the bias attribution seems to give cleaner explanations, and show much stronger focus on the head and beak. Overall, the bias attribution method tends to attribute in a more concentrated way, and often shows explanations about he DNNs complementary to the gradient information. The label of every image is shown in the leftmost column. The gradient attribution is the element-wise product between the linear model weight w and data point x. The "grad.norm." and "bias norm." columns show the attribution of gradient and bias normalized to the color range. The "grad.overlay" and "bias.overlay" show the 10% data features with the highest attribution magnitude of the original image. "bias\grad overlay" columns show the data features selected by the bias attribution and not selected by the gradient attribution. Bias1 correspond to the first proposed option of calculating the bias attribution score (Eq. FORMULA0 - FORMULA0), and bias2 (Eq.) correspond to the second proposed option. For both options of calculating the bias attribution score, the temperature parameter T is set to 1. For many cases, the bias attributions offer explanations complementary to the information provided by the gradient. For example, for the "bittern" image from ImageNet, BBp shows stronger attribution on the bird's head and body compared to the gradient method. For the "fire guard" image of ImageNet, BBp has clear attribution to the fire, in addition to the shape of the guard, while the gradient method only shows the shape of the guard. Similarly, for the "folding chair" of ImageNet, BBp shows clearer parts of the chair, while the gradient attribution shows less relevant features such as the wall.

represent the short summary for finding the corresponding scientific article (long)

Attribute the bias terms of deep neural networks to input features by a backpropagation-type algorithm; Generate complementary and highly interpretable explanations of DNNs in addition to gradient-based attributions.

Represent the short summary for finding the corresponding scientific article (long)

This work aims to provide quantitative answers to the relative importance of concepts of interest via concept activation vectors (CAV). In particular, this framework enables non-machine learning experts to express concepts of interest and test hypotheses using examples (e.g., a set of pictures that illustrate  the concept). We show that CAV can be learned given a relatively small set of examples. Hypothesis testing with CAV can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other sets of concepts. Interpreting networks with CAV does not require any retraining or modification of the network.


Question: Represent the text to find a very short summary (1-2 sentences max)

Question: This paper presents a method to autonomously find periodicities in a signal. It is based on the same idea of using Fourier Transform and autocorrelation function presented in Vlachos et al. 2005. While showing interesting this method does not perform well on noisy signals or signals with multiple periodicities. Thus, our method adds several new extra steps (hints clustering, filtering and detrending) to fix these issues. Experimental show that the proposed method outperforms the state of the art algorithms. A time series is defined by its 3 main components: the trend component, the periodic component and the random component. Trend analysis and prediction are topics that have been greatly studied BID10 and will not be treated in the article, therefore every time series will be assumed stationary regarding its mean and variance, so this study focus the periodic component. The ability to detect and find the main characteristic of this component is not as easy as the trend component. Yet, the ability to detect periodicities in a time series is essential to make precise forecasts. A periodicity is a pattern in a time series that occurs at regular time intervals. More precisely, the time series is said cyclical, if the time intervals at which the pattern repeats itself can't be precisely defined and is not constant. On the opposite, there are seasonal time series in which the pattern repeats itself at constant and well defined time intervals. Thus, cyclical patterns are more difficult to detect due to their inconsistency and the fact that they usually repeat themselves over large periods of time and therefore require more data to be identified. Nevertheless, seasonal patterns are very common in time series such as those related to human behaviour which usually have periodicities like hours and calendar (time of day, day of week, month of year). This kind of feature is well known and can be easily tested to see if they are beneficial or not. Unfortunately, when it comes to time series related to other phenomenons, the periodicities are not trivially found. For instance, tides level are multi-periodic time series correlated to both moon cycles and sun cycles; and females menstrual cycles are related to hormonal changes. The ability to detect periodicity in time series is fundamental when it comes to forecasting BID5. Once a periodic pattern has been detected, numerous techniques can be used to model this later and improve forecasts BID1. However, periodicities detection is not easy and has been greatly studied in the existing literature, but most of current techniques are unable to detect periodicities without the need of preprocessing data BID12 or have trouble detecting multiple periodicities BID11. This paper is organised as follow: we first present the Fourier transform and the Autoperiod algorithm BID11 used to detect periodicities in a signal. Then we propose a new fully automated method, named Clustered Filtered Detrended Autoperiod (CFD-Autoperiod), which also combines the advantages of frequency domain and time domain while being robust to noise and able to handle multi periodicities. Noise robustness is achieved using a density clustering on hints provided by the frequency analysis. Multi-periodicities are more precisely detected by both using detrending and filtering. Finally, we demonstrate that CFD-Autoperiod outperforms previous methods. Autocorrelation and Fourier transform are well known techniques used to find recurrent patterns in a given signal. The Fourier transform decomposes the original signal {s(t j)} j∈ [1,N] in a linear combination of complex sinusoids, also called a Fourier series. Let N be the number of frequency components of a signal, P the periodicity of the signal and c k the k th series coefficient then we have the Fourier series: DISPLAYFORM0 Thus, the amplitude and phase of each sinusoids correspond to the main frequencies contained within the signal. The Fourier transform can easily detect the presence of frequencies in a signal. However, if we need the corresponding periodicities from each frequency then we have to return from the frequency domain to the time domain. Let DF T be the Discrete Fourier Transform of a discrete signal {s(t j)}, then we can obtain the corresponding Periodogram P in the time domain as follow: DISPLAYFORM1 where f k = 2πk N correspond to the frequency captured by each component. However, in the frequency domain each bin is separated with a constant step of 1 N, whereas in the time domain bins size is N k(k+1), thus the range of periods is increasingly wider. Therefore, the Fourier transform resolution for long periods is not sufficient to provide an accurate estimation of the periodicity. Another way to find the dominant periodicities in a signal consists in calculating the autocorrelation function (ACF) of the given signal s(t). The autocorrelation is the correlation between the elements of a series and others from the same series separated from them by a given interval ∆t: DISPLAYFORM0 The ACF function provides a more accurate estimation of each periodicity, especially for longer periods as opposed to the Fourier transform BID11. However, it is not sufficient by itself due to the difficulty to select the most predominant peaks. Indeed, for a given periodicity p 1 the autocorrelation generates peaks for each p 1 multiple, hence the difficulty to select the relevant peaks when multiple periodicities composed a signal. A methodology combining both techniques advantages has been introduced by BID11. This method uses sequentially frequency domain (DFT) and time domain (ACF) in order to detect periodicity. The idea is to combine both methods in such a way that they complement each other. On the one hand, as mentioned earlier, due to its step inconstancy in the time domain, the Fourier transform resolution becomes insufficient to provide good estimations for long periods, where the autocorrelation has a constant resolution. On the other hand, according to BID11, it is difficult to correctly detect periodicities using only the autocorrelation function. Thus they proposed the following steps: first, noise is discarded from possible periodicity hints using a threshold on the Periodogram. Then, these hints are refined using the ACF function. If a periodicity hint lies on a local maximum then it can be validated, otherwise, if it lies on a local minimum this latter is discarded. On top of that, thanks to the ACF resolution, a gradient ascent is used to refine the remaining hints The Fourier Transform is used to select periodicity hints. To do so, we use the 99% confidence Li et al. BID11 technique to compute the threshold distinguishing periodicity hints from noise in the Fourier transform. Firstly, it is necessary to find the maximum amount of spectral power generated by the signal noise. Let be {s (t j)} j∈[1,N] a permuted sequence of a periodic sequence {s(t j)} j∈[1,N]. s should not exhibit any periodic pattern due to the random permutation process. Therefore, the maximal spectral power generated by s should not be higher than the spectral power generated by a true periodicity in s. Thus, we can use this value as a threshold to eliminate the noise. To provide a 99% confidence level, this process is repeated 100 times and the 99th largest value recorded is used as a threshold. Unfortunately, for a given periodicity in X, rather than finding an unique corresponding hint, spectral leakage may produce multiple hints near the true periodicity. This phenomenon is due to the finite resolution of the Fourier Transform and can only be avoided knowing in advance the true periodicities of the signal. Spectral leakage generates points with a spectral power higher than the threshold provided by the 99% confidence method FIG1 ) and therefore generate imprecise periodicity hints. The autocorrelation might filter most of them but every imprecise periodicity hint increase the probability of false positives, therefore it is interesting to reduce the number of periodicity hints in order to achieve a higher precision score. Knowing that the distribution of spectral leakage is more dense around the true periodicity, performing a density clustering over periodicity hints and using the ing centroids as periodicity hints can reduce the number of hints. A fundamental value in density clustering algorithms is the range in which it seeks for neighbours named. In our case, this value is not a constant because the accuracy of the hint is related to the resolution of the corresponding DFT bin size. A hint may have leaked from adjacent DFT bins, thus for a given hint of periodicity N/k, is set as the next bin value plus a constant width of 1 to avoid numerical issues when the difference from the current bin value to the next bin value is less than one: DISPLAYFORM0 The clustering is done by ascending periodicity order, hence a cluster made with small periodicities cannot be altered by bigger periodicity clusters. As shown in the FIG2, the density clustering performed in the GEF dataset BID3 drastically reduces the number of periodicity hints and the ing centroids are close to the true periodicities (24 and 168). Once the centroids have been found, they are used as periodicity hints during the validation step. For the validation step, a search interval for each periodicity hint is needed to check if this latter lies on a hill or a valley of the ACF. BID11 used the DFT bin size to define this search interval but in this study we propose a different approach. A periodicity N generates hills on the ACF at each multiple of N and valleys at each multiple of N 2. Therefore, we defined the search interval R for a periodicity hint N as follow: DISPLAYFORM0 Thereafter, a quadratic function is fitted to the ACR function in the search interval. In order to validate a hint, the function must have a negative second degree term and its derivative sign must change along the interval. The presence of multiple periodicities refutes the assumption that hills and valleys of the ACF are sufficient to validate or discard hints. Precisely, when validating a periodicity hint, correlations generated by both higher and lower frequencies than the hint can be problematic. These two problems are addressed in the following section. On the one hand, periodicities of higher frequencies induces sinusoidal correlations which may be in opposite phase with the correlation we are actually looking for (see FIG3 . Let s be a multiperiodic signal composed of periodicities P 1 and P 2 . Let P 1, a periodicity of length 20 and P 2, a periodicity of length 50. The periodicity P 1 produces on the signal ACF sinusoidal correlations of wavelength 20 and the periodicity P 2 produces sinusoidal correlations of wavelength 50. Thereby, at 50 lags on the ACF, the P 1 and P 2 periodicities will produce correlations in opposite phases and therefore nullify the hill at 50 used to validate or discard the periodicity hint P 2 . To tackle this issue, periodicity hints are analysed in ascending order. If a periodicity hint is validated, a lowpass filter with an adapted cutoff frequency is applied to the signal. Consequently, the following autocorrelations will be computed on this new signal. Thus, the ing autocorrelations will not exhibit any correlation induced by frequencies higher than the cutoff frequency of the lowpass filter. The cutoff frequency must be chosen carefully. Indeed, an ideal lowpass filter is characterised by a full transmission in the pass band, a complete attenuation in the stop band and an instant transition between the two bands. However, in practice, filters are only an approximation of this ideal filter and the higher the order of the filter is, the more the filter approximates the ideal filter. In our case, we are studying the periodicities in the signal, therefore, we want a filter with a frequency response as flat as possible to avoid any negative impact on the periodicity detection. Thereby, a Butterworth filter has been chosen due to its flat frequency response with no ripples in the passband nor in the stopband. However, a Butterworth filter, despite all the good properties, has a slow roll-off attenuating frequencies nearby the cutoff frequency. For the validation step, we do not want to attenuate the periodicity hint, therefore the cutoff frequency must not be exactly equal to the frequency of the hint. For a given periodicity N k, the frequency cutoff is equal to the previous bin value minus 1, to avoid the same numerical issues as described in the Density Clustering section: DISPLAYFORM0 On the other hand, low frequencies may induce a local trend in the autocorrelation that can be problematic when validating an hint. Indeed, in order to validate a periodicity hint, a quadratic function is fitted to the ACF in the search interval as mentioned in the subsection Hints Validation. Sadly, a trend in the search interval may prevent the derivative sign to switch FIG5), and therefore prevent the correct validation of the corresponding hint. Consequently, to avoid this situation, the ACF is detrended by subtracting the best fitted line in the following interval [0, N (k−1) + 1] for a given period hint N/k. Thus, the ing ACF does not exhibit any linear trend and therefore the fitted quadratic function is able to validate or discard hint efficiently. To evaluate the performances of the proposed method it is necessary to use time series datasets with periodicities. To do so, we perform our first evaluations on synthetic signals where the ground truth is known in order to compare raw performances and evaluations on real time series datasets. Signals of length 2000 with 1 to 3 periodicities have been generated. The periodicities have been chosen in the interval using a pseudo-random process. For multi-periodic signals, this pseudo-random process ensures that periodicities are not overlapping each others by checking that one is at least twice as bigger as the previous one. Finally, in order to compute precision and recall metrics, a validation criterion has been established. We assumed that a periodicity P d detected in a generated signal with a true periodicity P t is valid if: DISPLAYFORM0 The metrics have been consolidated over 500 iterations for each generated periodicity. As shown in Table 1, for a non multi-periodic signal, autoperiod and CFD-Autoperiod method achieve high precision scores whereas the Fourier Transform achieves a high recall but a really low precision score. Indeed, the Fourier Transform method does not filter the hints using the autocorrelation. Nevertheless, the autoperiod method did not detect every periodicities even for non multi-periodic signals autoperiod. This is likely due to the absence of density clustering and the narrow interval search to find the corresponding hill on the ACF. For multi-periodic signals, both recall and precision are drastically decreasing for the autoperiod method and as it can be observed, the detrending step and the use of a lowpass filter by the CFD-Autoperiod method lead to better scores. Regarding the Fourier Transform scores, due to the lack of the filtering step its recall is high but its precision score is always the lowest. Benchmarks have also been performed on synthetic signals generated via random process, without limitations on the periodicity values (Table 1). Naturally, the with an unique periodicity are similar. However, for multi-periodic signals the autoperiod and CFD-Autoperiod methods achieve lower scores. This is due to the fact that both methods use the autocorrelation to filter hints and this latter is not able to distinguish very close periodicities. Therefore, the use of autocorrelation as a validation step does not allow the detection of periodicities near each others. Nevertheless, in real datasets, most of the periodicities are sufficiently spaced to be detected by the autocorrelation function and thus remains efficient as a validation step. Benchmarks have also been performed on real datasets TAB3 ) and different types of time series have been chosen in order to test the validity of the proposed method.• GEF BID3 ): This dataset has been provided for the Global Energy Forecasting Competition 2014 (GEFCom2014) BID2, a probabilistic energy forecasting competition. The dataset is composed of 6 years of hourly load data. This time series is multi-periodic with the following periodicities: daily, weekly and bi-annual. The CFD-Autoperiod method has detected and validated 4 periodicities with 3 of them correct. Whereas the autoperiod has detected 5 periodicities with only 2 valid and has missed the long term bi-annual periodicity.• Great lakes: This dataset contains monthly water level of the 5 great lakes and is provided by the National Oceanic and Atmospheric Administration BID9. This time series is mono-periodic with a periodicity of 12 months. The autoperiod method has detected 4 different periodicities with only one correct. Among these latter, 24 and 72 periodicities were detected and are only ing correlations of the 12 periodicity. Whereas the CFD-Autoperiod has successfully filtered out the correlations of the 12 one. BID8 used this dataset as well but did not write the exact periodicities detected by their method. In their plots, the segmentation for both Ontario and Clair lakes does not correspond to a periodicity of 12.• Pseudo periodic (Keogh & Pazzani): These datasets contain 10 pseudo periodic time series generated from 10 different simulation runs. The data appears to be highly periodic, but never exactly repeats itself. BID8 did not write their exact but the segmentation shown on their plot seems to correspond to a detected periodicity of 155. The CFD-Autoperiod method found a periodicity of 144 and the exact true periodicity seems to be 142.• Boston Tides: This dataset contains water level records of Boston, MA from July 01 to August 31, with 6 minutes as sampling interval. It has been recently used by BID12 to evaluate their method. They successfully detected 2 periodicities but their method required a preprocessing step whereas the CFD-Autoperiod method does not require any. The first detected periodicity is 12,4 hours corresponding to the semi-diurnal constituent of 12 hours and 25.2 minutes. They have also detected 28,5 days and 29 days periodicities which correspond to a lunar month. The CFD-Autoperiod method detected a periodicity of 24 hours and 50 minutes whereas the autoperiod did not detect it. This value is interesting because it corresponds to the behaviour of a mixed tide (when there is a high high tide, a high low tide followed by a low high tide and a low low tide, in 24hour ans 50 minutes). However, it has not detected the lunar month periodicity but this might be due to the lack of data used. Indeed, BID12 used 2 months of data and the CFD-Autoperiod can only detect periodicities of a length inferior or equal to the half of the signal length. BID11. CFD-Autoperiod can be applied on noisy time series containing multiple periodicities and output raw periodicities that can later be refined by external domain specific knowledge (for instance 24h for human daily activities). One case not treated in this study concerns non-stationary series. A possible technique would consists in tracking the evolution of the periodicities through time and using a Kalman filter to track the apparition, disappearance or evolution of the detected periodicities. Using the confidence of the Kalman filter we could decide whether to continue considering the presence of a particular periodicity in the signal even if it is not detected for a while. This would strengthen the obtained by CFDAutoperiod and give more reliable periodicities. Thus, even more complex machine learning models can be built on top of them.

Represent the TLDR for finding the corresponding scientific article

This paper presents a method to autonomously find multiple periodicities in a signal, using FFT and ACF and add three news steps (clustering/filtering/detrending)

In this paper, a new method we call Centered Initial Attack (CIA) is provided. It insures by construction the maximum perturbation to be smaller than a threshold fixed beforehand, without the clipping process.