Question: Represent the text to find a very short summary (1-2 sentences max)

Question: Batch normalization (batch norm) is often used in an attempt to stabilize and accelerate training in deep neural networks. In many cases it indeed decreases the number of parameter updates required to achieve low training error. However, it also reduces robustness to small adversarial input perturbations and noise by double-digit percentages, as we show on five standard datasets. Furthermore, substituting weight decay for batch norm is sufficient to nullify the relationship between adversarial vulnerability and the input dimension. Our work is consistent with a mean-field analysis that found that batch norm causes exploding gradients. Batch norm is a standard component of modern deep neural networks, and tends to make the training process less sensitive to the choice of hyperparameters in many cases . While ease of training is desirable for model developers, an important concern among stakeholders is that of model robustness to plausible, previously unseen inputs during deployment. The adversarial examples phenomenon has exposed unstable predictions across state-of-the-art models . This has led to a variety of methods that aim to improve robustness, but doing so effectively remains a challenge BID0;; ). We believe that a prerequisite to developing methods that increase robustness is an understanding of factors that reduce it. Presented at the ICML 2019 Workshop on Identifying and Understanding Deep Learning Phenomena. Copyright 2019 by the author(s). Layer 2 Layer 14 Figure 1. Two mini-batches from the "Adversarial Spheres" dataset (2D), and their representations in a deep linear network with batch norm at initialization. Mini-batch membership is indicated by marker fill and class membership by colour. Each layer is projected to its two principal components. Classes are mixed by Layer 14.ing neural network architectures-that use batch norm-and patching them against specific attacks, e.g., through inclusion of adversarial examples during training (; ; ; Mądry et al., 2018). An implicit assumption is that batch norm itself does not reduce robustness -an assumption that we tested empirically and found to be invalid. In the original work that introduced batch norm, it was suggested that other forms of regularization can be turned down or disabled when using it without decreasing standard test accuracy. Robustness, however, is less forgiving: it is strongly impacted by the disparate mechanisms of various regularizers. The frequently made observation that adversarial vulnerability can scale with the input dimension (; ;) highlights the importance of identifying regularizers as more than merely a way to improve test accuracy. In particular, batch norm was a confounding factor in , making the of their initialization-time analysis hold after training. By adding 2 regularization and removing batch norm, we show that there is no inherent relationship between adversarial vulnerability and the input dimension. We briefly review how batch norm modifies the hidden layers' pre-activations h of a neural network. We use the notation of , where α is the index for a neuron, l for the layer, and i for a mini-batch of B samples from the dataset; N l denotes the number of neurons in layer l, W l is the matrix of weights and b l is the vector of biases that parametrize layer l. The batch mean is defined as µ α = 1 B i h αi, and the variance is σ DISPLAYFORM0 2. In the batch norm procedure, the mean µ α is subtracted from the pre-activation of each unit h l αi (consistent with), the is divided by the standard deviation σ α plus a small constant c to prevent division by zero, then scaled and shifted by the learned parameters γ α and β α, respectively. This is described in Eq., where a per-unit nonlinearity φ, e.g., ReLU, is applied after the normalization. DISPLAYFORM1 Note that this procedure fixes the first and second moments of all neurons α equally at initialization. This suppresses the information contained in these moments. Because batch norm induces a non-local batch-wise nonlinearity to each unit α, this loss of information cannot be recovered by the parameters γ α and β α.To understand how batch normalization is harmful, consider two mini-batches that differ by only a single example: due to the induced batch-wise nonlinearity, they will have different representations for each example . This difference is further amplified by stacking batch norm layers. Conversely, batch normalization of intermediate representations for two different inputs impair the ability to distinguish high-quality examples (as judged by an "oracle") that ought to be classified with a large prediction margin, from low-quality, i.e., more ambiguous, instances. We argue that this information loss and inability to maintain relative distances in the input space reduces adversarial as well as general robustness. Figure 1 shows a degradation of class-relevant input distances in a batch-normalized linear network on a 2D variant of the "Adversarial Spheres" dataset . We first evaluate the robustness (quantified as the drop in test accuracy under input perturbations) of convolutional networks, with and without batch norm, that were trained using standard procedures. The datasets -MNIST, SVHN, CIFAR-10, and ImageNet -were normalized to zero mean and unit variance. As a white-box adversarial attack we use projected gradient descent (PGD), ∞ -and 2 -norm variants, for its simplicity and ability to degrade performance with little perceptible 1 We add a ReLU nonlinearity when attempting to learn the binary classification task posed by in Appendix D, but the activations in the linear case give us pause. change to the input (Mądry et al., 2018). We run PGD for 20 iterations, with ∞ = 0.03 and a step size of ∞ /10 for SVHN, CIFAR-10, and ∞ = 0.01 for ImageNet. For PGD-2 we set 2 = ∞ √ d, where d is the input dimension. We report the test accuracy for additive Gaussian noise of zero mean and variance 1 /4, denoted as "Noise" , as well as the full CIFAR-10-C common corruption benchmark in Appendix C.We found these methods were sufficient to demonstrate a considerable disparity in robustness due to batch norm, but this is not intended as a formal security evaluation. All uncertainties are the standard error of the mean. For the SVHN dataset, models were trained by stochastic gradient descent (SGD) with momentum 0.9 for 50 epochs, with a batch size of 128 and initial learning rate of 0.01, which was dropped by a factor of ten at epochs 25 and 40. Trials were repeated over five random seeds. We show the of this experiment in TAB0, finding that despite batch norm increasing clean test accuracy by 1.86 ± 0.05%, it reduced test accuracy for additive noise by 5.5 ± 0.6%, for PGD-∞ by 17 ± 1%, and for PGD-2 by 20 ± 1%. Clean Noise PGD-∞ PGD-2 92.60 ± 0.04 83.6 ± 0.2 27.1 ± 0.3 22.0 ± 0.8 94.46 ± 0.02 78.1 ± 0.6 10 ± 1 1.6 ± 0.3For the CIFAR-10 experiments we trained models with a similar procedure as for SVHN, but with random 32 × 32 crops using four-pixel padding, and horizontal flips. In the first experiment, a basic comparison with and without batch norm shown in TAB1, we evaluated the best model in terms of test accuracy after training for 150 epochs with a fixed learning rate of 0.01. In this case, inclusion of batch norm reduces the clean generalization gap (difference between training and test accuracy) by 1.1 ± 0.2%. For additive noise, test accuracy drops by 6 ± 1%, and for PGD perturbations by 17.3 ± 0.7% and 5.9 ± 0.4% for ∞ and 2 variants, respectively. Very similar , presented in TAB2, are obtained on a new test set, CIFAR-10.1 v6 : batch norm slightly improves the clean test accuracy (by 2.0 ± 0.3%), but leads to a considerable drop in test accuracy for the cases with additive noise and the two 2 Each experiment has a unique uncertainty, hence the number of decimal places varies. Clean Noise PGD-∞ PGD-2 87.9 ± 0.1 78.9 ± 0.6 52.9 ± 0.6 65.6 ± 0.3 88.7 ± 0.1 73 ± 1 35.7 ± 0.3 59.7 ± 0.3 It has been suggested that one of the benefits of batch norm is that it facilitates training with a larger learning rate . We test this from a robustness perspective in an experiment summarized in Table 4, where the initial learning rate was increased to 0.1 when batch norm was used. We prolonged training for up to 350 epochs, and dropped the learning rate by a factor of ten at epoch 150 and 250 in both cases, which increases clean test accuracy relative to Table 2. The deepest model that was trainable using standard initialization without batch norm was VGG13. 3 None of the deeper models with batch norm recover the robustness of the most shallow, or same-depth equivalents without batch norm, nor does the higher learning rate in conjunction with batch norm improve robustness compared to baselines trained for the same number of epochs. Additional for deeper models on SVHN and CIFAR-10 can be found in Appendix A.We evaluated the robustness of pre-trained ImageNet models from the torchvision.models repository.4 Results are shown in TAB3, where batch norm improves top-5 accuracy on noise in some cases, but consistently reduces it by 8.54% to 11.00% (absolute) for PGD. The trends are the same for top-1 accuracy, only the absolute values were 3 For which one of ten random seeds failed to achieve better than chance accuracy on the training set, while others performed as expected. We report the first three successful runs for consistency with the other experiments.4 https://pytorch.org/docs/stable/ torchvision/models.html, v1.1.0. We show the effect of increasing the number of epochs on these trends in Appendix A.5. They also show in experiments that independence between vulnerability and the input dimension can be approximately recovered through adversarial training by projected gradient descent (PGD) (Mądry et al., 2018), with a modest tradeoff of clean accuracy. We show that this can be achieved by simpler means and with little to no trade-off through 2 weight decay, where the regularization constant λ corrects. We confirm that without regularization the loss does scale roughly as predicted: the predicted values lie between loss ratios obtained for = 0.05 and = 0.1 attacks for most image widths (see Table 4 of Appendix B). Training with 2 weight decay, however, we obtain adversarial test accuracy ratios of 0.98 ± 0.01, 0.96 ± 0.04, and 1.00 ± 0.03 and clean accuracy ratios of 0.999 ± 0.002, 0.996 ± 0.003, and 0.987 ± 0.004 for Next, we repeat this experiment with a two-hidden-layer ReLU MLP, with the number of hidden units equal to the half the input dimension, and optionally use one hidden layer with batch norm. 5 To evaluate robustness, 100 iterations of BIM-∞ were used with a step size of 1e-3, and Table 6. Evaluating the robustness of a MLP with and without batch norm. We observe a 61 ± 1% reduction in test accuracy due to batch norm for 97.95 ± 0.08 93.0 ± 0.4 66.7 ± 0.9 97.88 ± 0.09 76.6 ± 0.7 22.9 ± 0.7 56 98.19 ± 0.04 93.8 ± 0.1 53.2 ± 0.7 98.22 ± 0.02 79.3 ± 0.6 8.6 ± 0.8 84 98.27 ± 0.04 94.3 ± 0.1 47.6 ± 0.8 98.28 ± 0.05 80.5 ± 0.6 6.1 ± 0.5 Table 7. Evaluating the robustness of a MLP with 2 weight decay (same λ as for linear model, see Despite a difference in clean accuracy of only 0.08 ± 0.05%, Table 6 shows that for the original image resolution, batch norm reduced accuracy for noise by 16.4 ± 0.4%, and for BIM-∞ by 43.8 ± 0.5%. Robustness keeps decreasing as the image size increases, with the batch-normalized network having ∼ 40% less robustness to BIM and 13 − 16% less to noise at all sizes. We then apply the 2 regularization constants tuned for the respective input dimensions on the linear model to the ReLU MLP with no further adjustments. Table 7 shows that by adding sufficient 2 regularization (λ = 0.01) to recover the original (DISPLAYFORM0, no BN) accuracy for BIM of ≈ 66% when using batch norm, we induce a test error increase of 1.69 ± 0.01%, which is substantial on MNIST. Furthermore, using the same regularization constant without batch norm increases clean test accuracy by 1.39 ± 0.04%, and for the BIM-∞ perturbation by 21.7 ± 0.4%.Following the guidance in the original work on batch norm to the extreme (λ = 0): to reduce weight decay when using batch norm, accuracy for the ∞ = 0.1 perturbation is degraded by 79.3 ± 0.3% for √ d = 56, and 81.2 ± 0.2% for √ d = 84. In all cases, using batch norm greatly reduced test accuracy for noisy and adversarially perturbed inputs, while weight decay increased accuracy for such inputs. We found that there is no free lunch with batch norm: the accelerated training properties and occasionally higher clean test accuracy come at the cost of robustness, both to additive noise and for adversarial perturbations. We have shown that there is no inherent relationship between the input dimension and vulnerability. Our highlight the importance of identifying the disparate mechanisms of regularization techniques, especially when concerned about robustness. Bjorck, N., Gomes, C. P., Selman, B., and Weinberger, K. Q.Understanding This section contains supplementary explanations and to those of Section 3.A.1. Why the VGG Architecture?For SVHN and CIFAR-10 experiments, we selected the VGG family of models as a simple yet contemporary convolutional architecture whose development occurred independent of batch norm. This makes it suitable for a causal intervention, given that we want to study the effect of batch norm itself, and not batch norm + other architectural innovations + hyperparameter tuning. State-of-the-art architectures such as Inception, and ResNet whose development is more intimately linked with batch norm may be less suitable for this kind of analysis. The superior standard test accuracy of these models is somewhat moot given a trade-off between standard test accuracy and robustness, demonstrated in this work and elsewhere (; ; ;). Aside from these reasons, and provision of pre-trained variants on ImageNet with and without batch norm in torchvision.models for ease of reproducibility, this choice of architecture is arbitrary. We used the PGD implementation from Ding et al. FORMULA4 with settings as below. The pixel range was set to {±1} for SVHN, and {±2} for CIFAR-10 and ImageNet:from advertorch.attacks import LinfPGDAttack adversary = LinfPGDAttack(net, loss_fn=nn. CrossEntropyLoss(reduction="sum"), eps=0.03, nb_iter=20, eps_iter=0.003, rand_init=False, clip_min=-1.0, clip_max=1.0, targeted=False)We compared PGD using a step size of /10 to our own BIM implemenation with a step size of /20, for the same number of iterations. This reduces test accuracy for ∞ = 0.03 perturbations from 31.3 ± 0.2% for BIM to 27.1 ± 0.3% for PGD for the unnormalized VGG8 network, and from 15 ± 1% to 10 ± 1% for the batch-normalized network. The difference due to batch norm is identical in both cases: 17 ± 1%. Results were also consistent between PGD and BIM for ImageNet. We also tried increasing the number of PGD iterations for deeper networks. For VGG16 on CIFAR-10, using 40 iterations of PGD with a step size of ∞ /20, instead of 20 iterations with ∞ /10, reduced accuracy from 28.9 ± 0.2% to 28.5 ± 0.3%, a difference of only 0.4 ± 0.5%. Our first attempt to train VGG models on SVHN with more than 8 layers failed, therefore for a fair comparison we report the robustness of the deeper models that were only trainable by using batch norm in TAB6. None of these models obtained much better robustness in terms of PGD-2, although they did better for PGD-∞. Fixup initialization was recently proposed to reduce the use of normalization layers in deep residual networks (b). As a natural test we compare a WideResNet (28 layers, width factor 10) with Fixup versus the default architecture with batch norm. Note that the Fixup variant still contains one batch norm layer before the classification layer, but the number of batch norm layers is still greatly reduced. Fixup 94.6 ± 0.1 69.1 ± 1.1 20.3 ± 0.3 9.4 ± 0.2 87.5 ± 0.3 67.8 ± 0.9 BN 95.9 ± 0.1 57.6 ± 1.5 14.9 ± 0.6 8.3 ± 0.3 89.6 ± 0.2 58.3 ± 1.2We train WideResNets (WRN) with five unique seeds and show their test accuracies in TAB7. Consistent with Recht et al. FORMULA4, higher clean test accuracy on CIFAR-10, i.e. obtained by the WRN compared to VGG, translated to higher clean accuracy on CIFAR-10.1. However, these gains were wiped out by moderate Gaussian noise. VGG8 dramatically outperforms both WideResNet variants subject to noise, achieving 78.9 ± 0.6 vs. 69.1 ± 1.1. Unlike for VGG8, the WRN showed little generalization gap between noisy CIFAR-10 and 10.1 variants: 69.1 ± 1.1 is reasonably compatible with 67.8 ± 0.9, and 57.6 ± 1.5 with 58.3 ± 1.2. The Fixup variant improves accuracy by 11.6 ± 1.9% for noisy CIFAR-10, 9.5 ± 1.5% for noisy CIFAR-10.1, 5.4 ± 0.6% for PGD-∞, and 1.1 ± 0.4% for PGD-2.We believe our work serves as a compelling motivation for Fixup and other techniques that aim to reduce usage of batch normalization. The role of skip-connections should be isolated in future work since absolute values were consistently lower for residual networks. TAB0. ImageNet validation accuracy for adversarial examples transfered between VGG variants of various depths, indicated by number, with and without batch norm ("", ""). All adversarial examples were crafted with BIM-∞ using 10 steps and a step size of 5e-3, which is higher than for the white-box analysis to improve transferability. The BIM objective was simply misclassification, i.e., it was not a targeted attack. For efficiency reasons, we select 2048 samples from the validation set. Values along the diagonal in first two columns for Source = Target indicate white-box accuracy. The discrepancy between the in additive noise and for white-box BIM perturbations for ImageNet in Section 3 raises a natural question: Is gradient masking a factor influencing the success of the white-box on ImageNet? No, consistent with the white-box , when the target is unnormalized but the source is, top 1 accuracy is 10.5% − 16.4% higher, while top 5 accuracy is 5.3% − 7.5% higher, than vice versa. This can be observed in TAB0 by comparing the diagonals from lower left to upper right. When targeting an unnormalized model, we reduce top 1 accuracy by 16.5% − 20.4% using a source that is also unnormalized, compared to a difference of only 2.1% − 4.9% by matching batch normalized networks. This suggests that the features used by unnormalized networks are more stable than those of batch normalized networks. Unfortunately, the pre-trained ImageNet models provided by the PyTorch developers do not include hyperparameter settings or other training details. However, we believe that this speaks to the generality of the , i.e., that they are not sensitive to hyperparameters. ) respectively. The batch norm parameters γ and β were left as default, momentum disabled, and c = 1e-3. Each coordinate is first averaged over three seeds. Diamond shaped artefacts for unnormalized case indicate one of three seeds failed to train -note that we show an equivalent version of (a) with these outliers removed and additional batch sizes from 5-20 in FIG3. Best viewed in colour. In FIG5 we show that batch norm not only limits the maximum trainable depth, but robustness decreases with the batch size for depths that maintain test accuracy, at around 25 or fewer layers (in FIG5). Both clean accuracy and robustness showed little to no relationship with depth nor batch size in unnormalized networks. A few outliers are observed for unnormalized networks at large depths and batch size, which could be due to the reduced number of parameter update steps that from a higher batch size and fixed number of epochs .Note that in FIG5 (a) the bottom row-without batch norm-appears lighter than the equivalent plot above, with batch norm, indicating that unnormalized networks obtain less absolute peak accuracy than the batch normalized network. Given that the unnormalized networks take longer to converge, we prolong training for 40 total epochs. When they do converge, we see more configurations that achieve higher clean test accuracy than batch normalized networks in FIG5. Furthermore, good robustness can be experienced simultaneously with good clean test accuracy in unnormalized networks, whereas the regimes of good clean accuracy and robustness are still mostly non-overlapping in FIG5. Consider a logistic regression model with input x ∈ R d, labels y ∈ {±1}, parameterized by weights w, and bias b. Predictions are defined by o = w x + b and the model can be optimized by stochastic gradient descent (SGD) on the sigmoid cross entropy loss, which reduces to SGD on, where ζ is the softplus loss ζ(z) = log(1 + e −z): DISPLAYFORM0 We note that w x + b is a scaled, signed distance between x and the classification boundary defined by our model. If we define d(x) as the signed Euclidean distance between x and the boundary, then we have: w x + b = w 2 d(x). Hence, tency with the VGG8 experiment. Both models had already fit the training set by this point.minimizing is equivalent to minimizing DISPLAYFORM1 We define the scaled softplus loss as DISPLAYFORM2 and note that adding a 2 regularization term in, ing in, can be understood as a way of controlling the scaling of the softplus function: To test this theory empirically we study a single linear layer on variants of MNIST of increasing input dimension, where the "core idea" from is exact. Clearly, this model is too simple to obtain competitive test accuracy, but this is a helpful first step that will be subsequently extended to ReLU networks. The model was trained by SGD for 50 epochs with a constant learning rate of 1e-2 and a batch size of 128. In TAB0 we show that increasing the input dimension by resizing MNIST from 28 × 28 to various resolutions with PIL.Image. NEAREST interpolation increases adversarial vulnerability in terms of accuracy and loss. Furthermore, the "adversarial damage", which is predicted to grow like √ d by Theorem 4 of , falls in between that obtained empirically for = 0.05 and = 0.1 for all image widths except for 112, which experiences slightly more damage than anticipated. note that independence between vulnerability and the input dimension can be recovered through adversarial example augmented training by projected gradient descent (PGD), with a small trade-off in terms of standard test accuracy. We find that the same can be achieved through a much simpler approach: 2 weight decay, with λ chosen to correct for the loss scaling. This way we recover input dimension invariant vulnerability with little degradation of test accuracy, e.g., see the = 0.1 accuracy ratio of 1.00 ± 0.03 with 2 for TAB0 compared to 0.10 ± 0.09 without. DISPLAYFORM3 DISPLAYFORM4 Compared to PGD training, weight decay regularization i) does not have an arbitrary hyperparameter that ignores intersample distances, ii) does not prolong training by a multiplicative factor given by the number of steps in the inner loop, and 3) is less attack-specific. Thus, we do not use adversarially augmented training because we wish to convey a notion of robustness to unseen attacks and common corruptions. Furthermore, enforcing robustness to -perturbations may increase vulnerability to invariance-based examples, where semantic changes are made to the input thus changing the Oracle label, but not the classifier's prediction . Our models trained with weight decay obtained 12% higher accuracy (86 vs. 74 correct) compared to batch norm on a small sample of 100 ∞ invariance-based MNIST examples. 8 We make primary use of traditional p perturbations as they are well studied in the literature and straightforward to compute, but solely defending against these is not the end goal. A more detailed comparison between adversarial training and weight decay can be found in. The scaling of the loss function mechanism of weight decay is complementary to other mechanisms identified in the literature recently, for instance that it also increases the effective learning rate (van ; a). Our are consistent with these works in that weight decay reduces the generalization gap, even in batch-normalized networks where it is presumed to have no effect. Given that batch norm is not typically used on the last layer, the loss scaling mechanism persists in this setting although to a lesser degree. We evaluated robustness on the common corruptions and perturbations benchmarks . Common corruptions are 19 types of real-world effects that can be grouped into four categories: "noise", "blur", "weather", and "digital". Each corruption has five "severity" or intensity levels. These are applied to the test sets of CIFAR-10 and ImageNet, denoted CIFAR-10-C and ImageNet-C respectively. When reporting the mean corruption error (mCE), we average over intensity levels for each corruption, then over all corruptions. We outline the for two VGG variants and a WideResNet on CIFAR-10-C, trained from scratch independently over three and five random seeds, respectively. The most important are also summarized in TAB0.For VGG8 batch norm increased the error rate for all noise variants, at every intensity level. The mean generalization gaps 91.74 ± 0.02 64.5 ± 0.8 63.3 ± 0.3 70.9 ± 0.4 71.5 ± 0.5 65.3 ± 0.6 93.0 ± 0.1 43.6 ± 1.2 49.7 ± 0.5 56.8 ± 0.9 60.4 ± 0.7 67.7 ± 0.5 WRN-28-10 F 94.6 ± 0.1 63.3 ± 0.9 66.7 ± 0.9 71.7 ± 0.7 73.5 ± 0.6 81.2 ± 0.7 95.9 ± 0.1 51.2 ± 2.7 56.0 ± 2.7 63.0 ± 2.5 66.6 ± 2.5 86.0 ± 0.9 for noise were: Gaussian-9.2 ± 1.9%, Impulse-7.5 ± 0.8%, Shot-5.6 ± 1.6%, and Speckle-4.5 ± 1.6%. The next most impactful corruptions were: Contrast-4.4 ± 1.3%, Spatter-2.4 ± 0.7%, JPEG-2.0 ± 0.4%, and Pixelate-1.3 ± 0.5%.Results for the remaining corruptions were a coin toss as to whether batch norm improved or degraded robustness, as the random error was in the same ballpark as the difference being measured. These were: Weather-Brightness, Frost, Snow, and Saturate; Blur-Defocus, Gaussian, Glass, Zoom and Motion; and Elastic transformation. Averaging over all corruptions we get an mCE gap of 1.9 ± 0.9% due to batch norm, or a loss of accuracy from 72.9 ± 0.7% to 71.0 ± 0.6%.VGG13 were mostly consistent with VGG8: batch norm increased the error rate for all noise variants, at every intensity level. Particularly notable, the generalization gap enlarged to 26 − 28% for Gaussian noise at severity levels 3, 4, and 5; and 17%+ for Impulse noise at levels 4 and 5. Averaging over all levels, we have gaps for noise variants of: Gaussian-20.9 ± 1.4%, Impulse-13.6 ± 0.6%, Shot-14.1 ± 1.0%, and Speckle-11.1 ± 0.8%. Robustness to the other corruptions seemed to benefit from the slightly higher clean test accuracy of 1.3 ± 0.1% due to batch norm for VGG13. The remaining generalization gaps varied from (negative) 0.2 ± 1.3% for Zoom blur, to 2.9 ± 0.6% for Pixelate. Overall mCE was reduced by 2.0 ± 0.3% for the unnormalized network. For a WideResNet 28-10 (WRN) using "Fixup" initialization (b) to reduce the use of batch norm, the mCE was similarly reduced by 1.6 ± 0.4%. Unpacking each category, the mean generalization gaps for noise were: Gaussian-12.1 ± 2.8%, Impulse-10.7 ± 2.9%, Shot-8.7 ± 2.6%, and Speckle-6.9 ± 2.6%. Note that the large uncertainty for these measurements is due to high variance for the model with batch norm, on average 2.3% versus 0.7% for Fixup. JPEG compression was next at 4.6 ± 0.3%.Interestingly, some corruptions that led to a positive gap for VGG8 showed a negative gap for the WRN, i.e., batch norm improved accuracy to: Contrast-4.9 ± 1.1%, Snow-2.8 ± 0.4%, Spatter-2.3 ± 0.8%. These were the same corruptions for which VGG13 lost, or did not improve its robustness when batch norm was removed, hence why we believe these correlate with standard test accuracy (highest for WRN). Visually, these corruptions appear to preserve texture information. Conversely, noise is applied in a spatially global way that disproportionately degrades these textures, emphasizing shapes and edges. It is now well known that modern CNNs trained on standard datasets have a propensity to rely excessively on texture rather than shape cues . The WRN obtains ≈ 0 training error and is in our view over-fitted; CIFAR-10 is known to be difficult to learn robustly given few samples . The "Adversarial Spheres" dataset contains points sampled uniformly from the surfaces of two concentric n-dimensional spheres with radii R = 1 and R = 1.3 respectively, and the classification task is to attribute a given point to the inner or outer sphere. We consider the case n = 2, that is, datapoints from two concentric circles. This simple problem poses a challenge to the conventional wisdom regarding batch norm: not only does batch norm harm robustness, it makes training less stable. In Figure 6 we show that, using the same architecture as in , the batch-normalized network is highly sensitive to the learning rate η. We use SGD instead of Adam to avoid introducing unnecessary complexity, and especially since SGD has been shown to converge to the maximum-margin solution for linearly separable data (Soudry et al., Figure 5 . Two mini-batches from the "Adversarial Spheres" dataset (2D variant), and their representations in a deep linear network at initialization time (a) with batch norm and (b) without batch norm. Mini-batch membership is indicated by marker fill and class membership by colour. Each layer is projected to its two principal components. In (b) we scale both components by a factor of 100, as the dynamic range decreases with depth under default initialization. We observe in (a) that some samples are already overlapping at Layer 2, and classes are mixed at Layer 14.(a) (b) Figure 6. We train the same two-hidden-layer fully connected network of width 1000 units using ReLU activations and a mini-batch size of 50 on a 2D variant of the "Adversarial Spheres" binary classification problem . Dashed lines denote the model with batch norm. The batch-normalized model fails to train for a learning rate of η = 0.01, which otherwise converges quickly for the unnormalized equivalent. We repeat the experiment over five random seeds, shaded regions indicate a 95% confidence interval.2018). We use a finite dataset of 500 samples from N (0, I) projected onto the circles. The unormalized network achieves zero training error for η up to 0.1 (not shown), whereas the batch-normalized network is already untrainable at η = 0.01. To evaluate robustness, we sample 10,000 test points from the same distribution for each class (20k total), and apply noise drawn from N (0, 0.005 × I). We evaluate only the models that could be trained to 100% training accuracy with the smaller learning rate of η = 0.001. The model with batch norm classifies 94.83% of these points correctly, while the unnormalized net obtains 96.06%. We show a qualitative aspect of batch norm by visualizing the activations of the penultimate hidden layer in a fully-connected network (a) without and (b) with batch norm over the course of 500 epochs. In the unnormalized network 7(a), all data points are overlapping at initialization. Over the first ≈ 20 epochs, the points spread further apart (middle plot) and begin to form clusters. In the final stage (epochs ≈ 300 − 500), the clusters become tighter. When we introduce two batch-norm layers in the network, placing them before the visualized layer, the activation patterns display notable differences, as shown in FIG9 (b): i) at initialization, all data points are spread out, allowing easier partitioning into clusters and thus facilitating faster training; ii) the clusters are more stationary, and the stages of cluster formation and tightening are not as strictly separated; iii) the inter-cluster distance and the clusters themselves are larger, indicating that the decision boundary is more sensitive to small input variations.

Batch normalization reduces adversarial robustness, as well as general robustness in many cases, particularly to noise corruptions.

We apply a model-agnostic defense strategy against adversarial examples and achieve 60% white-box accuracy and 90% black-box accuracy against major attack algorithms.


Question: Represent the scientific article to find a short TLDR

Question: This paper presents preliminary ideas of our work for auto- mated learning of Hierarchical Goal Networks in nondeter- ministic domains. We are currently implementing the ideas expressed in this paper. Many domains are amenable to hierarchical problem-solving representations whereby complex problems are represented and solved at different levels of abstraction. Examples include some navigation tasks where hierarchical A* has been shown to be a natural solution solving the navigation problem over different levels of abstraction BID29 BID66; dividing a reinforcement learning task into subtasks where policy control is learned for subproblems and combined to form a solution for the overall problem BID9 BID10 BID12; abstraction planning, where concrete problems are transformed into abstract problem formulations, these abstract problems are solved as abstract plans, and in turn these abstract plans are refined into concrete solutions BID31 BID4; and hierarchical task network (HTN) planning where complex tasks are recursively decomposed into simpler tasks BID8 BID68 BID15 BID43. These paradigms have in common a divideand-conquer method to problem solving that is amenable to stratified representation of the subproblems. Among the various formalisms, HTN planning has been a recurrent research focus over the years. An HTN planner formulates a plan using actions and HTN methods. The latter describe how and when to reduce complex tasks into simpler subtasks. HTN methods are used to recursively decompose tasks until so-called primitive tasks are reached corresponding to actions that can be performed directly in the world. The HTN planners SHOP and SHOP2 have routinely demonstrated impressive gains in performance (runtime and otherwise) over standard planners. The primary reason for these performance gains is because of the capability of HTN planners to exploit domain-specific knowledge BID67. HTNs provide a natuCopyright c 2019, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. ral knowledge-modeling representation for many domains, including military planning BID38 BID40, strategy formulation in computer games BID23 BID22, manufacturing processes BID47 BID61, project planning BID62 BID63, story-telling BID6, web service composition, and UAV planning BID20 Despite these successes, HTN planning suffers from a representational flaw centered around the notion of task. A task is informally defined as a description of an activity to be performed (e.g., find the location of robot r15) (e.g., the task "dislodge red team from Magan hill" in some adversarial game) and syntactically represented as a logical atom (e.g., (locate r15)). (e.g., "(dislodge redteam Magan)"). Beyond this syntax, there is no explicit semantics of what tasks actually mean in HTN representations. HTN planners obviate this issue by requiring that a complete collection of tasks and methods is given, one that decomposes every complex task in every plausible situation. However, the knowledge engineering effort of creating a complete set of tasks and methods can be significant BID17. Furthermore, researchers have pointed out that the lack of tasks' semantics make using HTNs problematic for execution monitoring problems BID13 BID14 ). Unlike goals, which are conditions that can be evaluated against the current state of the world, tasks have no explicit semantics other than decomposing them using methods. For example, suppose that a team of robots is trying to locate r15 and, using HTN planning, it generates a plan calling for the different robots to ascertain r15's location. While executing the plan generate a complex plan in a gaming task to dislodge red team from Magan hill, the HTN planner might set a complex plan to cutoff access to Magan, surround it, weaken the defenders with artillery fire and then proceed to assault it. If sometime while executing the plan, the opponent abandons the hill, the plan would continue to be executed despite the fact that the task is already achieved. This is due to the lack of task semantics, so their fulfillment cannot be checked against the current state; instead their fulfillment is only guaranteed when the execution of the generated plans is completed. Hierarchical Goal Networks (HGNs) solve these limitations by representing goals (not tasks) at all echelons of the hierarchy BID56. Hence, goal fulfillment can be directly checked against the current state. In particular, even when a goal g is decomposed into other goals (i.e., in HGN, HGN methods decompose goals into subgoals), the question if the goal is achieved can be answered directly by checking if it is valid in the current state. So in the previous example, when the opponent abandons the hill, an agent executing the plan knows this goal has been achieved regardless of how far it got into executing the said plan. Another advantage of HGNs is that it relaxes the complete domain requirement of HTN planning BID57; in HTN planning a complete set of HTN methods for each task is needed to generate plans. Even if the HGN methods are incomplete, it is still possible to generate solution plans by falling back to standard planning techniques such as heuristic planning BID24 to achieve any open goals. Nevertheless, having a collection of well-crafted HGN methods can lead to significant improvement in performance over standard planning techniques BID59.When the HGN domain is complete (i.e., there is no need to revert to standard planning techniques to solve any problem in the domain), its expressiveness is equivalent to Simple Hierarchical Ordered Planning BID59, which is the particular variant of HTN planning used by the widely used SHOP and SHOP2 BID45 ) HTN planners. SHOP requires the user to specify a total order of the tasks; SHOP2 drops this requirement allowing partial-order between the tasks BID44. Both have the same representation capabilities although SHOP2 is usually preferred since it doesn't force the user to provide a total order for the method's subtasks BID44.In this work, we propose the automated learning of HGNs for ND domains extending our previous work on learning HTNs for deterministic domains BID21. While work exists on learning goal hierarchies BID53 BID32 BID49, these works are based on formalisms that have more limited representations than HGNs and in fact predate them. Aside from HGNs, researchers have explored other ways to address the limitation associated with the lack of tasks' semantics. For instance, TMKs (Task-Method-Knowledge models) require not only the tasks and methods to be given but also the semantics of the tasks themselves as (preconditions,effects) pairs BID41 BID42. While this solves the issue with the lack of tasks' semantics it may exacerbate the knowledge engineering requirement of HTNs: the knowledge engineer must not only encode the methods and tasks but also must encode their semantics and ensure that the methods are consistent with the given tasks' semantics. To deal with incomplete HTN domains, researchers have proposed translating the methods into a collection of actions so that standard planning techniques can be used BID1 ). There are two limitations with this approach. First, HTN planning is strictly more expressive than standard planning BID15, hence the translation will be incomplete in many domains. Second, for domains when translating methods into actions is possible, it may in exponentiallymany actions on the number of methods. HGNs are more in line with efforts combining HTN and standard planning approaches BID30 BID17; the main difference is that HGNs eliminate the use of tasks all-together while still preserving the expressiveness of Simple Hierarchical Ordered Planning BID59.The problem of learning hierarchical planning knowledge has been a frequent research subject over the years. For example, ICARUS (Choi and Langley 2005) learns HTN methods by using skills (i.e., abstract definitions of semantics of complex actions) represented as Horn clauses. The crucial step is a teleoreactive process where planning is used to fill gaps in the HTN planning knowledge. For example, if the learned HTN knowledge is able to get a package from an starting location to a location L1 and the HTN knowledge is also able to get the package from a location L2 to its destination, but there is no HTN knowledge on how to get the package from L1 to L2, then an standard planner is used to generate a plan to get the package from L1 to L2 and skills are used to learn new HTN methods from the plan generated to fill the gap on how to get from L1 to L2.Another example is HTN-Maker (Hogg, Muñoz-Avila, and Kuter 2008). HTN-Maker uses task semantics defined as (preconditions,effects) pairs, exactly like TMKs mentioned before, to identify sequences of contiguous actions in the input plan trace where the preconditions and effects are met. Task hierarchies are learned when an action sequence is identified as achieving a task and the action sequence is a sub-sequence of another larger action sequence achieving another task. This includes the special case when the sub-sequence and the sequence achieve the same task. In such a situation recursion is learned. HTN-Maker learns incrementally after each training case is given. HTNLearn BID70 transforms the input traces into a constraint satisfaction problem. Like HTN-Maker, it also assumes (preconditions,effects) as the task semantics to be given as input. HTNLearn process the input traces converting them into constraints. For example, if a literal p is observed before an action a and a is a candidate first sub-task for a method m, then a constraint c is added indicating that p is a precondition of m. These constrains are solved by a MAXSAT solver, which returns the truth value for each constraint. For example, if c is true then p is added as a precondition of m. As a of the MAXSAT process, HTNLearn is not able to converge to a 100% correct domain (the evaluation of HTNLearn computes the error rates in the learned domain).Similar to HTN planning, hierarchical decompositions have been used in hierarchical reinforcement learning BID50 BID10. The hierarchical structure of the reinforcement learning problem is analogous to an instance of the decomposition tree that an HTN planner might generate. Given this hierarchical structure, hierarchical reinforcement learners perform value-function composition for a task based on the value functions learned over its subtasks recursively. However, the possible hierarchical decompositions must be provided in advance. Hierarchical goal networks (HGNs) BID56 are an alternative representation formalism to HTNs. In HGNs, goals, instead of tasks, are decomposed at every level of the hierarchy. HGN methods have the same fo.rm as HTN methods but instead of decomposing a task, they decompose a goal; analogously instead of subtasks, HGN methods have subgoals. If the domain description is incomplete, HGNs can fall back to STRIPS planners to fill gaps in the domain. On the other hand, total-order HGNs are as expressive as totalorder HTNs BID59 and its partial-order variant ) is as expressive as partial-order HTNs.Inductive learning has been used to learn rules indicating goal-subgoal relations in X-learn BID53. This is akin to learning macro-operators BID39 BID5; the learned rules and macro-operators provide search control knowledge to reach the goals more rapidly but they don't add expressibility to standard planning. SOAR learns goal-subgoal relations BID32. It uses as input annotated behavior trace structures, indicating the decisions that led to the plans; this is used to generate a goal-subgoal relations. Another work on learning goal-subgoal relations is reported in BID49 ). It uses case-based learning techniques to store goal-subgoal relations, which are then reused by using similarity metrics. These works assume some form of the input traces, unstructured in BID49 ) and structured in BID32, to be annotated with the subgoals as they are accomplished in the traces. In our proposed work, the input traces are not annotated and, more importantly, we are learning HGNs. Goal regression techniques have been used to generate a plan starting from the goals that must be achieved BID52 BID36. The of goal regression can be seen as a hierarchy recursively generated by indicating for each goal what subgoals must be achieved. The goal-subgoal relations ing from goal regression are a direct consequence of the domain's operators: the goals are effects of the operators and the preconditions are the subgoals. In contrast, in a HGN, the hierarchies of goals represent relations between the HGN methods and are not necessarily implied directly from the actions. Making an analogy with HTN methods, HGN methods capture additional domain-specific knowledge BID45 or generate plans with desirable properties (e.g., taking into account quality considerations) again not explicitly represented in the actions BID27.Work on learning hierarchical plan knowledge is related to learning of context-free grammars (CFGs), which aims at eliciting a finite set of production rules from a finite set of strings BID48 BID55 ). The precise definition of the learning problem varies constraining the ing CFG by, among others, providing a target function (e.g., obtaining a CFG with the minimum number of production rules) or assuming that negative examples (i.e., strings that must not be generated by the CFG) are given. To learn CFGs, algorithms search for production rules that generate the training set (and none of the negative examples when provided). Grammar learning is exploited by the Greedy Structure Hypothesizer (GSH) BID35, which uses probabilistic context-free grammars learning techniques to learn a hierarchical structure of the input plan traces. GSH doesnt learn preconditions since its goals are not to generate the grammars for planning but to reflect users preferences. The difference between learning CFG and learning hierarchical planning knowledge is twofold. First, characters that form a string have no meaning. In contrast, actions in a given plan are defined by their preconditions and effects. This means that plausible strings generated by the grammars may be invalid when viewed as plans. Second, learning HGNs requires not only learning the task decomposition but also the preconditions. This is an important difference: HTNs are strictly more expressive than CFGs BID16. Intuitively, HTNs are akin to context-sensitive grammars in that they constraint when a decomposition can take place. Context-sensitive grammars are also strictly more expressive then CFGs BID60.Finally, as we will see in the next the next section, our proposed work is related to the notion of planning landmarks BID25. Given a planning problem P, defined as a triple (s 0, g,A), indicating the initial state, the goals and the actions respectively, a planning landmark is either an action a ∈ A, or state atom p ∈ s (s is an state, represented as a collection of atoms) that occurs in any solution plan trace solving P. Given the problem description P, planning systems can identify automatically landmarks for P. Planning landmarks have been widely used for automated planning ing in planners such as LAMA BID54 and the HGN planner GoDel BID57 ). We want to learn HGNs for fully observable nondeterministic (FOND) planning BID19 Speck, Ortlieb, and Mattmüller 2015; BID69. In such domains, actions may have multiple outcomes. For example, in the Minecraft simulation, when a character swings a sword to hit a monster, there are two possible outcomes: either the sword hits the monster or the monster parries it and the sword doesn't hit anything. As discussed before, HTN learners require the tasks semantics to be given either as Horn clauses defining the tasks or as (preconditions,effects) pairs. The latter is used, for example, in the nondeterministic HTN learner ND-HTNMaker, a state-of-the-art HTN learner, to pinpoint locations in the traces where the various tasks are fulfilled. ND-HTNMaker enforces a right recursive structure: exactly one primitive task followed by none or exactly one compound task. The main objective of enforcing this right recursive structure is to deal with nondeterminism: if, for example, the character swings the sword (e.g., a primitive task), the follow-up compound task handles the nondeterminism: one method decomposing a compound task t will simply perform the action to swing at the monster followed by t, thereby ensuring that method can be triggered as many times as needed until the monster is hit (and dies). Other methods decomposing t handle the case when the monster has been dealt with (e.g., a method handling the case when "character next to a dead monster"). This ensures that methods learned by HTN-Maker are provable correct BID26. Correctness can be loosely defined as follows: any solution generated by a sound nondeterministic HTN planner such as ND-SHOP BID33 ) using the learned methods and the nondeterministic actions is also a solution when using the nondeterministic actions (i.e., without the methods).Like in the deterministic case, the inputs will be a collection of actions A and a collection of traces s 0 a 0 s 1 a 1... a n s n+1, where each a i ∈ A. Only this time, any action a i ∈ A may have multiple outcomes; so each occurrence of a i in the input traces will reflect one such outcome. Planning in nondeterministic domains requires to account for all possible outcomes. As such, BID7 proposed a categorization of solutions for nondeterministic domains. It distinguishes between weak, strong cyclic and strong solutions for a problem (s 0, g,A). A solution is represented as a policy π: S → A, a mapping from the possible states in the world S to actions A, indicating for any given state s ∈ S, what action π(s) to take. Given a policy π, an execution trace is any sequence s 0 π(s 0) s 1 π(s 1)... π(s n) s n+1, where s i is an state that can be reached from state s i−1 after applying action π(s i−1).A solution policy π is weak if there exists an execution trace from s 0 to a state satisfying g. Weak solutions guarantee that a goal state can be successfully reached sometimes. For example, in the Minecraft simulation, a policy that assumes a computer-controlled character will always hit any monster it encounters when swinging the sword is considered a weak solution. In particular, this solution would not account for the situation when the monster parries the character's sword attack; e.g., the monster might counter-attack and disable the agent and the agent has not planned what to do in such a situation. Under the fairness assumption, stating that "every action executed infinitely often will exhibit all its effects infinitely often" (D'Ippolito, Rodrıguez, and Sardina 2018), a solution π is either strong cyclic or strong if every terminal state entails the goals and for every state s that the agent might finds itself in after executing π from s 0, there exists an execution trace from the state s to a state satisfying g. The difference is that in strong cyclic solutions the same state might be visited more than once whereas in strong solutions this never happens. For example, a strong cyclic solution might have the character swing the sword against the monster and if the monster parries the attack, the character takes an step back to avoid the monster's counter-attack and step towards the monster while taking another swing at it; this can be repeated as many times as needed until the monster dies. Strong solutions are ideal since they never visit the same state but in some domains they might not exists. For instance, there are no strong solutions in the Minecraft simulation mentioned as the monster can repeatedly parry the character's attacks. The same occurs in the robot navigation domain BID7 ), created to model nondeterminism. In this domain a robot is navigating between offices and when it encounters a closed door for an office it wants to access, the robot will open it. There is an another agent acting in the environment that closes doors at random. So the robot might need to repeatedly execute the action to open the same door. Solving nondeterministic planning problems is difficult because of what has been dubbed the explosion of states as a of the nondeterminism BID19. One demonstrated way to counter this is by adding domain-specific knowledge as described in BID33 ). While the algorithm described is generic for a variety of ways to encode the domain-specific knowledge, it showcases hierarchical planning techniques outperforming an state-of-the-art nondeterministic planner in some domains including the robot navigation domain. The show either speedups of several orders of magnitude or the ability to solve problems of sizes, measured by the number of goals to achieve, previously impossible to solve. Relation to probabilistic domains. In this work we are neither assuming a probability distribution over the possible actions' outcomes to be given nor we aim to learn such a distribution. Once an HGN domain is learned, hierarchical reinforcement learning techniques BID10 can be used to learn a probability distribution over the various possible goal decompositions and exploit the learned distribution during problem solving as done in BID27.We propose to learn bridge atoms and their hierarchical structure with the important constraint that the learned hierarchical structure must encode the domain's nondeterminism in a sound way. For instance, the nondeterministic version of the logistics transportation domain in BID26 ) extends the deterministic version as follows: when loading a package p into vehicle v in a location l there are two possible outcomes: either p is inside v or p is still at l (i.e., the load action failed). Regardless of possibly repeating the same action multiple times, traces will bring the package to the airport, transport it by air to the destination city, and deliver it. So the kinds of decompositions we are aiming to learn should also work on nondeterministic domains; on the other hand a learned hierarchy would be unsound if, for example, it assumes that the load truck action always succeeds and immediately proceeds to deliver the package to an airport. This will lead to weak solutions. To correctly handle nondeterminism, we propose forcing a right-recursive structure on lower echelons of the learned HGNs. This takes care of the nondeterminism and combine well with the higher decompositions. For instance, in the transportation domain we identified a goal g airp, for the package p reaching the airport, identified as a bridge atom, and then have all methods achieving g airp be right recursive; e.g., methods of the form (: method g airp prec (g g airp) <), where g is some intermediate goal such as loading the package into a vehicle. Our aim is the automated learning of HGN methods. This includes learning the goals, the goal-subgoal structure of the HGN methods and their applicability conditions. Specifically, the learning problem can be defined as follows: given a set of actions A and a collection of traces Π generated using actions in A, to obtain a collection of HGN methods. A collection of methods M is correct if given any (initial state, goal) pair (s 0, g), and any solution plan π generated by a sound HGN planner using M and A, π is a correct plan solving the planning problem (s 0, g,A). An HGN method m is a construct of the form (:method head(m) preconditions(m) subgoals(m) <(m)) corresponding to the goal decomposed by m (called the head of m), the preconditions for applying m and the subgoals decomposing head(m). Figure 1 shows an example of an HGN method in the logistics transportation domain BID65. (the question marks indicate variables. It recursively decomposes the goal of delivering ? pack1 into ? loc2 into three subgoals: delivering? pack1 to the airport? airp1 in the same city as its current location? loc1, delivering? pack1 to the airport? airp2 in the same city as the destination location? loc2, and recursively achieve the head goal):Head: Package-delivery Preconditions: (at ?pack ? loc1 ? city1) (airport ? airp1 ? city1) (airport ? airp2 ?city2) (location ? loc2 ? city2) (= ?city1 ?city2) Subgoals: g 1: (package-at ?pack ? airp1) g 2:(package-at ?pack1 ? airp2) g 3:(package-at ?pack ? loc2 ? city2) Constraints: g 1 < g 3, g 2 < g 3 Figure 1: Example of an HGN method in the logistics transportation domain. The question marks indicate variables. The goal achieved by the method is the last subgoal, g 3. It recursively decomposes the goal of delivering? pack into? loc2 into three subgoals: delivering? pack1 to the airport? airp1 in the same city as its current location? loc1, delivering? pack to the airport? airp2 in the same city as the destination location? loc2, and g 3 is to be achieved after g 1 and g 2 are achieved. HGNs planners BID57 BID56 maintain a list G = g 1,..., g n of open goals (i.e., goals to achieve). Planning follows a recursive procedure, starting with π =, choosing the first element, g 1, in G, and either applying an HGN method m decomposing g 1 into m's subgoals g 1,..., g k, concatenating m's subgoals into G (i.e, G = g 1, . . ., g k, g 1, . . ., g n are the new open goals), or applying an action a ∈ A achieving g, appending a to π (i.e., π ← π · a) and removing g from G. In either case it will check if the preconditions of m (respectively, a) are satisfied in the current state. When a is applied, the current state is transformed in the usual way BID18. When G = ∅, π is returned. HGN planners extend this basic procedure to allow the use of standard planning techniques to achieve open goals and to enable a partial ordering between the methods' subgoals. the planner picks the first goal in G without predecessors. For example, in Figure 1, the user may define the constraints: g 1 < g 3, g 2 < g 3, and the planner instead of always picking the first subgoal in G, it picks the first subgoal without predecessors. We propose transforming the problem of identifying the goals and learning their hierarchical relation into the problem of finding relations between word embeddings extracted from text. Specifically, we propose viewing the collection of input traces Π as text: each plan trace π = s 0 a 0 s 1 a 1... a n s n+1 is viewed as a sentence w 1 w 2... w m; each action a i and each atom in s j is viewed as a word w k in the sentence. The order of the plan elements in each trace is preserved (we use the term plan element to refer to both atoms and actions): the word w j = a i appears before the word w j = p, for every p ∈ s i+1. In turn, every w j appears before w j = a i+1.Word embeddings are vectors representing words in a multi-dimensional vector space BID3 BID2. There are a number of algorithms to do this translation BID37 BID51. They have in common that they represent vector similarity based on the cooccurrence of words in the text. That is, words that tend to occur near one another will have similar vector representations. In our preliminary work we used Word2Vec BID37 ) (i.e., Word-Neighboring Word), a widely used algorithm for generating word embeddings. Word2Vec uses a shallow neural network, consisting of a single hidden layer, to compute these vector representation; it computes a context window W consisting of k contiguous words and trains the network using each word w ∈ W (i.e., W is w's context). The window W is "moved" one word at the time through the text further training the network each time. Training is repeated with windows of size i = {1, 2, . . . k}. For this reason, Word2Vec is said to use "dynamic windows". In Word2Vec, similarity is computed with the cosine similarity, sim C, because it measures how close is the orientation of the ing vectors, which are distributed in such a way that words frequently co-occurring in the context windows have similar orientation whereas those that co-occur less frequently will have a dissimilar orientation. There are two particularities of the change of representation from plan elements to word embeddings that is particularly suitable for our purposes: first the procedure is unsupervised. This means in our case that we do not have to annotate the traces with additional information such as where the goals are been achieved in the traces. Second, vector representations are generated based on the context in which they occur (e.g., the dynamic window W in Word2Vec). In our case, the vector representations of the plan elements will be generated based on their proximity to other plan elements in the traces. These vectors can be clustered together into plan elements that are close to one another. Our working hypothesis, supported by previous work BID21, is that what we call bridge atoms, are ideal candidates for goals. Given two clusters of plan element embeddings, A and B, a bridge atom, bridge AB, is an atom in either A or B that is most similar to the plan elements in the other set. Establishing a bridge atom hierarchy is a recursive process that first requires calculating the bridge atom of a corpus, splitting each text around the bridge atom so that each text in the corpus becomes two new texts, before and after the bridge atom, and then repeating the procedure on the ing sub-corpora. The procedure for find a bridge atom for a corpora is as follows. We train a Word2Vec model on the corpus to determine the word vectors and cluster them with Hierarchical Agglomerative Clustering. Currently we limit the number of clusters to two, although later research may explore how to determine the number of clusters from the structure of the traces. We determine the cosine distance of each atom in a cluster to each atom in the other and average them together for each atom, selecting the word with the shortest average distance, DISPLAYFORM0 where dist C is the cosine distance between the vector representations of two atoms. If an action is selected as the bridge atom, we instead use in its place the atom describing one of its goals. As previously stated, by splitting each trace around the bridge atom, we can form two new sub-corpora, one from the section of each trace before the bridge atom and one from the section after the bridge atom. Then we recursively perform the procedure for bridge atom selection on each new corpora, keeping track of the hierarchical relationship of each subcorpora to the other corpora. If during the division process, a section of a trace becomes shorter than some threshold, we discard it from the sub-corpora. Progress along any branch of recursion halts once there are insufficient traces in a subcorpus for training. We use the hierarchy of bridge atoms as a guide for building a set of hierarchical methods. At the lowest level of division are single-action or short multi-action sections of the traces. Each of these sections will become a method with a single goal (an effect of an action) or a method with multiple goals (one for each of the actions). Each of these methods have two subgoals: one for the subsection of trace before a bridge atom and another one for the trace after that bridge atom. Each action is annotated with its preconditions. The preconditions of a method can be extrapolated from the preconditions of the actions into which it decomposes by regressing over the actions of that section of the plan trace in reverse, collecting the action preconditions and removing from the preconditions any atom which is in the effects of chronologically-preceding action. We are using a variant of the Pyhop HTN planner (https: //bitbucket.org/dananau/pyhop). Our variant introduces nondeterminism in the actions and generates solution policies as described in BID33.Our experiments use a nondeterministic variant of the logistics domain BID64 ). In the domain, packages must be relocated from one location to another. Trucks transport packages within cities, and airplanes transport packages between cities via airports. Nondeterminism is introduced via the load and unload operators, which have two outcomes, success (the package is loaded onto/unload from the specified vehicle) or failure (the package does not move from its original location). We have also added rockets that transport packages between cities on different planets via launchpads. All traces demonstrate a plan for achieving the same goal, the relocation of a package from a location in one city on the starting planet to a location in a city on the destination planet. To ensure that Word2Vec can identify common bridge atoms across the corpus, the package and each location must have the same name in all traces. Although Word2Vec typically works best on a corpus of thousands of texts or more, we are able to learn reasonable bridge atoms from hundreds of texts by increasing the number of epochs and lowering learning rate. For our problem design, a reasonable first bridge atom is one that involves the package and a rocket or launchpad, as transporting the package from the start planet to the destination planet marks the halfway point in the traces. From a corpus of 700 traces, with 1000 epochs and a learning rate of 0.00025, our first bridge atom is the action unload(package, rocket).Because word embeddings are sensitive to word context, the trace structure influences the bridge atom hierarchy. Which atoms are included in the trace and where they are included is important. We are experimenting with two different variants of state expression within traces. In one variant, we list each action preceded by its deletelist and followed by its addlist. If an atom occurs in the addlist of one action and the deletelist of the subsequent action, that atom will only appear in the addlist of the first action. In another variant, we list actions preceded by their preconditions and followed by their effects. In both variants, atoms are listed alphabetically.

Represent the TLDR for finding the corresponding scientific article

Represent the short summary for finding the corresponding scientific article (long)

This paper introduces partial grounding to tackle the problem that arises when the full grounding process, i.e., the translation of a PDDL input task into a ground representation like STRIPS, is infeasible due to memory or time constraints.