Question: Represent the text to find a very short summary (1-2 sentences max)

Question: Determining the number of latent dimensions is a ubiquitous problem in machine learning. In this study, we introduce a novel method that relies on SVD to discover the number of latent dimensions. The general principle behind the method is to compare the curve of singular values of the SVD decomposition of a data set with the randomized data set curve. The inferred number of latent dimensions corresponds to the crossing point of the two curves. To evaluate our methodology, we compare it with competing methods such as Kaisers eigenvalue-greater-than-one rule (K1), Parallel Analysis (PA), Velicers MAP test (Minimum Average Partial). We also compare our method with the Silhouette Width (SW) technique which is used in different clustering methods to determine the optimal number of clusters. The on synthetic data shows that the Parallel Analysis and our method have similar and more accurate than the other methods, and that our methods is slightly better than the Parallel Analysis method for the sparse data sets. The problem of determining the number of latent dimensions, or latent factors, is ubiquitous in a number of non supervised learning approaches. Matrix factorization techniques are good examples where we need to determine the number of latent dimensions prior to the learning phase. Non linear models such as LDA BID1 and neural networks also face the issue of stating the number of topics and nodes to include in the model before running an analysis over a data set, a problem that is akin to finding the number of latent factors. We propose a new method to estimate the number of latent dimensions that relies on the Singular Value Decomposition (SVD) and on a process of comparison of the singular values from the original matrix data with those from from bootstraped samples of the this matrix, whilst the name given to this method, Bootstrap SVD (BSVD). We compare the method to mainstream latent dimensions estimate techniques and over a space of dense vs. sparse matrices for Normal and non-Normal distributions. This paper is organized as follow. First, we outline some of best known methods and the related works in the next section. Then we explain our algorithm BSVD in section 3. The experiments are presented in section 4 and the and discussion are reported in section 5. And finally of the study is given in section 6. The problem of finding the number of latent factors in a data set dates back to early work by BID4. It extends to a large array of fields including psychology and social science BID2, bioinformatics BID8, information retrieval BID25, and of course statistical learning (, chapter 14). A systematic review on latent variables and its applications can be found in BID15; BID12 BID21.In the following, we briefly explain some of the best known techniques of factor analysis method to decide the number of factors to retain as mentioned in BID14 such as Kaiser's eigenvalue-greater-than-one rule (K1), Parallel Analysis, Cattell's Scree test, Velicer's MAP test (Minimum Average Partial). Moreover, we outline SW technique which is used in different clustering methods to determine the optimal number of clusters. The K1-Kaiser method was first introduced by BID4 and later extended and popularized by BID11. The method an early strategy used to detect the number of factors to retain by considering. It relies on the eigenvalues of the correlation matrix of the observed factors an stipulates that the number of eigenvalues greater than one corresponds to the number of latent factors to retain. Despite its simplicity, some researches consider it unreliable BID14 BID17 BID3. We will nevertheless consider it in our comparison experiments, given that it is a classic method and the first that introduced the use of eigenvectors of the correlation matrix for determining the number of latent factors. Note that a variant of this method was introduced by BID26 that showed improvements but still lacked behind PA (§2.2 and MAP ( §2.4) and will therefore not be included in the experiment. Parallel Analysis is also based on the correlation matrix between the observed factors. It uses bootstrapping on the correlation matrix and then averages the eigenvalues over the bootstrap runs. Eigenvalues greater than the average data set eigenvalue are kept. BID17; BID14. This strategy was proposed by BID9. BID26 showed that PA attempt improves over the Eigenvalue-greater-than-one rule. Several researchers found this method appropriate and more accurate in determining the number of factors to retain BID16 BID17. We will see that it PA a close relationship with BSVD and this is corroborated by the closeness of the . Cattell's Scree test, also known as the "elbow" approach, is a graphical representation method to display the number of components to retain. Scree test is a subjective method which sorts the eigenvalues in decreasing order and shows it on the vertical axis; and the number of components in horizontal axis BID14. In this strategy, we need to find where the Scree happened and the components on the left side of the slope should be retained. BID5 used the Scree test plot and mention that the method fails when elbow cannot be found. Moreover, BID23 mentions the Scree test as one of the graphical tests to finding the number of latent variables, they specify that researchers should utilize of the ruler to plot a line across the elbow and then they could keep all the components above it. Despite, the various criticism to use this method by BID27 and BID10, it is one of the most popular methods to find the number of important factors to retain MAP approach is based on PCA and relies on the series of partial correlation matrices to define the number of significant factors to retain BID17 BID14 BID27 BID18. This approach is introduced by BID24. In general, statisticians agree that the MAP and PA are the two techniques which are reliable solution to extract the number of factors to retain with the reasonable Ledesma Clustering is generally tought of as a means to reduce the number of data points, but it can be considered for dimensionality reduction technique, namely by using the cluster's centroid and each point's distance from them as a means to define a new space, which was shown to be more effective than SVD under certain conditions . Therefore, a method to determine the optimal number of clusters can provide another means to determine the dimensionality of a data set. PAM method is one of the popular technique to automatically determine the optimal number of clusters with the Silouhette (SW) technique BID13. The number of clusters computed by SW is associated with the number of latent dimensions in the dataset. SVD is a well known matrix factorization technique that decomposes the original matrix, R, into the product of two eigenvector matrices, the eigenvectors of the cross-product of the rows and columns, and a of the diagonal matrix of their common singular values. DISPLAYFORM0 where U and V are orthogonal, and Σ is a diagonal matrix with positive real values. The singular values represent the importance of the eigenvectors, ordered by decreasing values. The BSVD method determines the number of dimensions as the point where the singular values of Σ cross the singular values Σ B of a randomized (through bootstrap sampling) matrix R B. An interpretation of this crossing point where Σ and Σ B meet is that the remaining singular values are no more due to influential factors, at least in a linear framework. An example of this can be seen in figure 2 for a data set that was generated using 9 dimensions (vertical line) with uniform distributed values. In this case, it is also easy to tell the number from the elbow at dimensions= 9.The bootstrapped samples R B are simply generated through random sampling with replacement of the values of R.In the next section, we look at the details of generating data sets and the experiments. We evaluate the ability of BSVD to identify the number of latent dimensions by using synthetic data. Although the use of synthetic data limits the generalizability of our to the real world by making strong assumptions on the data, it remains the best validation methodology given that we know the ground truth behind the synthetic data and we can control the sparsity and the underlying distributions of each of the observed variables in order to explore this space of conditions. 4.1 DATA SETS 4.1.1 SYNTHETIC DATA The synthetic data is generated by sampling from distributions to create two matrices, P and Q. Then, R is obtained by the product P · Q plus Gaussian noise. We use two types of distributions, the normal (Gaussian) with mean= 0 and standard deviation= 1, and the uniform distribution with mean= 2.5 to generate the columns of P and rows of Q. The choice of 2.5 is inspired from rating-type of data found in recommender systems. The Gaussian noise added to P · Q corresponds to one standard deviation of R with mean= 0.All R matrices are of size 150 × 240 and we explore the latent dimensions from 2 to 24.In figure 1, we illustrate an example of the generated non-normal data set with size of 5 × 6 and latent dimension 3.Data sets of different density are generated, since sparsity is a constraint that we often have to deal with in fields such as recommender systems (rating matrices) and natural language processing (term-document matrices). Sparsity is created by randomly selecting the missing value cells. In order to capture the behavior of our method when we face a sparse matrix, we employ the algorithm with different percentages of sparseness to the data set (see algorithm 1). Then, we compare the with the existing mentioned approaches in Table 2. To do so, we follow the next steps for each iteration of latent dimension (j): 1) We apply a different percentage of sparseness (j) from 10: 90 to our data set with random selection.2) We impute each missing value by the average of the mean of corresponding row and column.3) Apply BSVD. And for each latent dimension (j); record the in each iteration of k. Compute the average accuracy of each method, when (j) terminate. Figure 5 displays the accuracy of all the methods in the non-normal sparse data set with latent dimension (j) equal to 2. We repeat the previous experiments on the generated simulated random data set with a normal distribution. The BSVD algorithm is compared with Horn's PA and K1 implementations from BID7. Moreover, we used of Very Simple Structure(VSS) and PAM packages of R to have the outcome of MAP and SW methods respectively. According to the of provided experiments in the tables 1 and 2, we could show that our method has a better performance than those mentioned especially in the sparse data sets. Our empirical experiments demonstrate that on the dense data sets; the accuracy of BSVD and PA is equal and better than the other approaches. But when we apply a different percentage of sparseness to our data sets, our method is more precise. In the figures 3 and 4, we display the behavior of each method in the dense and sparse data sets. Figure 3 depicts the average accuracy of all methods in the dense data sets with normal and nonnormal distribution. It shows that MAP method in the dense data set with normal or non-normal distribution has the same accuracy. Additionally, SW technique performs better with the face of the dense data set with non-normal distribution, while K1 has an extreme behavior in the nonnormal data set. Moreover, BSVD, PA and K1 are more precise in the dense data set with normal distribution. Figure 4 shows the sparse data sets with normal and non-normal distribution. It demonstrates that BSVD, PA, and K1 have better accuracy in the sparse data set with normal distribution but MAP and SW are on the contrary. Figure 5 shows the average accuracy of all the methods in in different level of sparsity over the non normal sparse data set with latent dimensions (j) equal to 2. The error bars shows the variance of the observations after repeating the algorithm 25 times. Based on the of these experiments we can conclude that our approach (BSVD) is better than the presented methods especially in the sparse data sets. To show if the outcome is statistically significant and is not by chance, we apply t-test between our method and PA. We considered the p values less than or equal to 0.05 as a significant . To do so, we consider a sample of latent dimensions (j = {2, 3, 5, 8, 15}) and we repeat twenty-five times the mentioned experiments on the sparse data sets with normal and non-normal distribution, and record the . Then we apply t-test between BSVD and PA. In this evaluation the null hypothesis (H0) state that µ SV D = µ P A and if the H0 is rejected, we could conclude that the obtained are not by chance and our method is better than PA. TAB1 contain p values of the sparse and dense data sets with normal and non-normal distribution respectively. The first row of each table with 0% of sparsity indicate to the dense data sets. TAB1 shows more constant behavior, and implies that by increasing sparsity in the sparse data set with normal distribution, BSVD yeilds a significantly better . But table 4 that shows the of non-normal sparse data set is hard to interpret. Because the green cells are not affected by increasing sparsity. We can sum up with that the seems to be significant with increasing the sparsity. In general, according to the tables 3 and 4, the difference between our method and PA seems to be statistically significant by increasing the percentage of sparsity. The objective of our study was to introduce a new method to find the number of latent dimensions using SVD which we inspired from PA. We employ our method on simulated data sets with normal and non-normal distribution whereas are dense or sparse and compared with the present methods such as PA, MAP, K1, and SW. According to the mentioned experiments and the reported in the table 1, BSVD and PA have the same accuracy and better than the other presented methods in the dense data sets. But our method has a better in the sparse data sets which is shown in the table 2. We applied t-test on the sample of latent dimensions (j) between BSVD and PA to demonstrate if the is statistically significant or not. The in the tables (3 and 4) demonstrate that in the sparse data sets with increasing the sparsity, our method seems to be significantly better than the other methods. Our method performance is limited to the presented experiments and data sets. If we want to generalize the method, We need to see the behavior of the algorithm when we have a more complex data set.step a. Generating the matrices x and y with the sizes of 6 × 3 and 5 × 3.

represent the short summary for finding the corresponding scientific article (long).

In this study, we introduce a novel method that relies on SVD to discover the number of latent dimensions.

Represent the TLDR for finding the corresponding scientific article

This work aims to provide quantitative answers to the relative importance of concepts of interest via concept activation vectors (CAV). In particular, this framework enables non-machine learning experts to express concepts of interest and test hypotheses using examples (e.g., a set of pictures that illustrate  the concept). We show that CAV can be learned given a relatively small set of examples. Hypothesis testing with CAV can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other sets of concepts. Interpreting networks with CAV does not require any retraining or modification of the network.


Question: Represent the article for finding a single-sentence TLDR

Question: Deep learning models are often sensitive to adversarial attacks, where carefully-designed input samples can cause the system to produce incorrect decisions. Here we focus on the problem of detecting attacks, rather than robust classification, since detecting that an attack occurs may be even more important than avoiding misclassification. We build on advances in explainability, where activity-map-like explanations are used to justify and validate decisions, by highlighting features that are involved with a classification decision. The key observation is that it is hard to create explanations for incorrect decisions. We propose EXAID, a novel attack-detection approach, which uses model explainability to identify images whose explanations are inconsistent with the predicted class. Specifically, we use SHAP, which uses Shapley values in the space of the input image, to identify which input features contribute to a class decision. Interestingly, this approach does not require to modify the attacked model, and it can be applied without modelling a specific attack. It can therefore be applied successfully to detect unfamiliar attacks, that were unknown at the time the detection model was designed. We evaluate EXAID on two benchmark datasets CIFAR-10 and SVHN, and against three leading attack techniques, FGSM, PGD and C&W. We find that EXAID improves over the SoTA detection methods by a large margin across a wide range of noise levels, improving detection from 70% to over 90% for small perturbations. Machine learning systems can be tricked to make incorrect decisions, when presented with samples that were slightly perturbed, but in special, adversarial ways . This sensitivity, by now widely studied, can hurt networks regardless of the application domain, and can be applied without knowledge of the model . Detecting such adversarial attacks is currently a key problem in machine learning. To motivate our approach, consider how most conferences decide on which papers get accepted for publication. Human classifiers, known as reviewers, make classification decisions, but unfortunately these are notoriously noisy. To verify that their decision are sensible, reviewers are also asked to explain and justify their decision. Then, a second classifier, known as an area-chair or an editor, examines the classification, together with the explanation and the paper itself, to verify that the explanation supports the decision. If the justification is not valid, the review may be discounted or ignored. In this paper, we build on a similar intuition: Explaining a decision can reduce misclassification. Clearly, the analogy is not perfect, since unlike human reviewers, for deep models we do not have trustworthy methods to provide high level semantic explanation of decisions. Instead, we study below the effect of using the wider concept of explanation on detecting incorrect decisions, and in particular given adversarial samples that are designed to confuse a classifier. The key idea is that different classes have different explaining features, and that by probing explanations, one can detect classification decisions that are inconsistent with the explanation. For example, if an image is classified as a dog, but has an explanation that gives high weight to a striped pattern, it is more likely that the classification is incorrect. We focus here on the problem of detecting adversarial samples, rather than developing a system that provides robust classifications under adversarial attacks. This is because in many cases we are interested to detect that an attack occurs, even if we cannot automatically correct the decision. The key idea in detecting adversarial attacks, is to identify cases where the network behaves differently than when presented with untainted inputs, and previous methods focused on various different aspects of the network to recognize such different behaviours;;;;;;. To detect these differences, here we build on recent work in explainability Lundberg & Lee (2017b). The key intuition is that explainability algorithms are designed to point to input features that are the reason for making a decision. Even though leading explainability methods are still mostly based on high-order correlations and not necessarily identify purely causal features, they often yield features that people identify as causal (a). Explainability therefore operates directly against the aim of adversarial methods, which perturb images in directions that are not causal for a class. The is that detection methods based on explainability holds the promise to work particularly well with adversarial perturbations that lead to nonsensical classification decisions. There is second major reason why using explainable features for adversarial detection is promising. Explainable features are designed to explain the classification decision of a classifier trained on non-modified (normal) data. As a , they are independent of any specific adversarial attack. Some previous methods are based on learning the statistical abnormalities of the added perturbation. This makes them sensitive to the specific perturbation characteristics, which change from one attack method to another, or with change of hyperparameters. Instead, explainability models can be agnostic of the particular perturbation method. The challenge in detecting adversarial attacks becomes more severe when the perturbations of the input samples are small. Techniques like C&W Carlini & Wagner (2017b) can adaptively select the noise level for a given input, to reach the smallest perturbation that causes incorrect classification. It is therefore particularly important to design detection methods that can operate in the regime of small perturbations. Explanation-based detection is inherently less sensitive to the magnitude of the perturbation, because it focuses on those input features that explain a decision for a given class. In this paper we describe an EXAID (EXplAIn-then-Detect), an explanation-based method to detect adversarial attacks. It is designed to capture low-noise perturbations from unknown attacks, by building an explanation model per-class that can be trained without access to any adversarial samples. Our novel contributions are as follows: We describe a new approach to detect adversarial attacks using explainability techniques. We study the effect of negative sampling techniques to train such detectors. We also study the robustness of this approach in the regime of low-noise (small perturbations). Finally, we show that the new detection provides state-of-the-art defense against the three leading attacks (FGSM, PGD, CW) both for known attacks and in the setting of detecting unfamiliar attacks. Explainable AI. Several methods have been recently proposed to address black box decisions of AI systems. LIME is based on locally approximating the model around a given prediction with a simple interpretable model (e.g. a decision tree). DeepLIFT uses a modified version of back propagation to compute the contribution of each input feature to the output. SHAP (a) approximates (a) (b) Figure 1: Illustration of EXAID. First, an image is classified by a standard image classification system like ResNet. Then, an explanation is created based on the image, the network activations and the network output. Finally, a detector checks if the generated explanation is consistent with the predicted label. (a) An image of an owl is correctly classified, and the produced explanation is consistent with the label "owl". (b) An image of peacock is perturbed and used as an attack. It is falsely classified as an owl, and is detected as adversarial because its explanation is inconsistent with the predicted label. the Shapley values of input features, which were derived in cooperative game theory to distribute the total gains to the players. Specifically, for explaining deep neural networks, SHAP use a variant of DeepLIFT as a approximation for Shapley values. The literature on adversarial attacks is vast. We focus here on three high-performing adversarial attacks which are relevant for the experiments. Each of the three represents a group of attacks that share the same main idea. This attack by creates a perturbation by "moving" an example one step in the direction of the gradient. Let c be the true class of x and J(C, x, c) be the loss function used to train our deep neural network C. The perturbation is computed as a sign of the model's loss function gradient ∆x = * sign(∇ x J(C, x, c)), where ranging from 0.0 to 1.0. The parameter controls the magnitude perturbation and can be thought as the noise-level of the adversarial sample. Projected Gradient Decent (PGD). suggested to improve FGSM, in the following way. One can interpret FGSM as a one-step scheme for maximizing the inner part of the saddle point formulation. A more powerful adversary will be a multi-step variant, which essentially applies projected gradient descent on the negative loss function Carlini and Wagner (C&W). Carlini & Wagner (2017b) employed an optimization algorithm to seek the smallest perturbation that enables an adversarial example to fool the classifier. As showed in (a), this attack is considered to be one of the most powerful attacks, and therefore is a common baseline. When designing attacks, previous studies took into account various factors: the probability that the attack is successful, the effect on the appearance of a perturbed image, and the time it takes to run the attack. The above three methods prioritize these aspects differently, reaching different tradeoff operating points. Specifically, FGSM is usually faster and the C&W attack yields less-visible perturbation of the input images. Several previous techniques have been proposed to detect adversarial examples. measured the effect of quantization and smoothing of the image on the network classification, both parameterized as a function of image entropy. suggested to reduce the degrees of freedom of the input space by applying transformations like quantization and smoothing, and then compare model predication before and after each transformation. measures the characteristics of the region surrounding a reference example. models the distribution of ac-tivation at the hidden layers of the classifier, using a Gaussian mixture model, and feeds the likelihood to a classifier. models the changes in the labels of a k-nn for each activation layer in the base model. propose using a new loss in training, which encourages the neural network to learn latent representations that better distinguish adversarial examples from normal ones. models the statistical robustness of log-odds to perturbations, for normal and adversarial examples. Generally speaking, these methods assume that adversarial examples differ intrinsically from natural images, either in the sample space or because the perturbation affects propagation of activity in the neural network. Some of those methods require modifying the base model. Very recently, described an explanation-based approach to detection, related to the current paper. 3 EXAID: EXplAIn then Detect EXAID consists of two components: Explain. Create per-class explanations for both correct predictions and incorrect ones. Detect. Train a binary classifier to decide if an explanation is consistent with the class decision. These two components are schematically shown in Figure 1. The first step in EXAID implements an explanation model. Given a pretrained classifier that may be attacked, we used an explainability model to extract explanations for every sample classified by the model. The explanation model can take as input the raw input image, as well as the whole base model architecture and weights, and produces an explanation in the terms of the input features. Formally it is a function that maps a sample and a classifier, and its prediction into explanation space E: (x, f theta (x)) → R n, where f theta is a classification model producing a predicted label y = output(f (x)). Since our goal is to learn which explanations are typical for each class, we collected both positive explanations -applying an explanation model to a correct prediction of the network, and negative explanations -corresponding to incorrect predictions of the model. Creating positive explanations E (x i, y i) is usually straight forward, as one simply applies the explanation model on each sample that was correctly classified f (x i) = y i. More care should be given to creating negative explanations. We consider three types of negative explanations: wrong negatives, adversarial negatives and other-class negatives. First, one may collect samples (x i, y i) where the model made an incorrect decision f (x i) = y i, and collect their explanations E (x i, f (y i)). We name these wrong negatives. For models that are well trained, the number of these explanations is small. Furthermore, not all classes are confused by other classes, and only some classes may lead to explanations of some other classes. Second, one can employ an adversarial attack on the training data and collect negative explanations of adversarially perturbed samples. We name them adversarial negatives. As with wrong negatives, these explanations correspond to cases where the model made an incorrect decision, but unlike wrong negatives the explanations may have a different distribution, because the input was designed to confuse the network. Even if the specific type of adversarial attack is not known, these samples may be useful because they are based on fooled decisions and may reflect typical patterns of adversarial examples. However, training against an incorrect attack may cause overfitting to a specific type of attack and hurt detection accuracy. Third, for every labeled sample (x i, y i), we produce explanations E (x i, y) for all incorrect classes y ∈ Y, y = y i. For example, for a car image correctly classified as a car, we produce explanations for classes like dogs and cats. These explanations are used as other-class negatives for the correct class y i. As an explainable AI approach we used SHAP deep explainer. As shown in (b) SHAP is considered a leading explainer, providing explanations that have stronger agreement with human explanations than other methods. We therefore believe it is likely to capture the "correct" features by which people make labeling decisions. In addition, Lundberg & Lee (2017b) has shown that SHAP is the only explainer that has both local accuracy and consistency, which are desirable properties. Given a set of positive and negative explanations per class, we train a deep binary classifier per class, to detect explanations that are inconsistent with model predictions. Note that in this settings, it is natural to train a detector as a binary multiclass multi-label classifier, and not as a multiclass classifier, because we wish to condition the decision on the prediction of the image classifier. When training the detector, one may consider two learning setups, aiming to protect against unknown-attacks, or against familiar attacks. It appears as if defending against a known attack would be an easier task, because one may learn the properties of the attack. Unfortunately, since new attacks can be easily designed, it is highly desirable to devise generic defenses. We address this topic by controlling the data that is used for training the detector. Specifically, we consider two variants of EXAID. EXAID familiar. During training, the binary detector is presented with adversarial negatives. It can therefore learn a distribution of explanations ing from a specific adversarial attack. Specifically, we trained using high-noise FGSM. EXAID unknown. The binary detector is not presented with any adversarial negatives during training. The only negative explanation the classifier trained on are other-class negatives and wrong negatives. Below we tested both variants on the known attack (FGSM) and on unfamiliar attacks (PGD, C&W). We evaluated EXAID on two benchmark data sets, in the task of attack detection. Our code will be available at https://github.com/[anonymous-author]/EXAID. We evaluated EXAID on two standard benchmarks: CIFAR10 (Krizhevsky et al.) and SVHN . As Carlini & Wagner (2017a) showed, MNIST is not a good dataset for evaluating adversarial defences. This is probably due to the fact that it is a low-dimension dataset, making it easier to detect changes an attacker made to the image. (a) show their on CIFAR-10. In order to show the validity of our on more than one dataset, we also used SVHN that has similar complexity. The CIFAR-10 dataset consists of 60,000 32x32 colour images in 10 classes, with 6,000 images per class. There are 50,000 training images and 10,000 test images. The SVHN dataset is obtained from house numbers in Google Street View images. It consists of more than 600,000 32x32 colour images in 10 classes. While similar in flavor to MNIST, it comes from a significantly more diverse distribution. We used the 73,257 digits provided for training and the 26,032 digits for testing. For both CIFAR-10 and SVHN we used a pretrained Resnet34 as a base model. To train the EXAID detector we extract positive explanation, wrong negative and other-class negative from natural images as described in algorithm 1. The EXAID-unknown model was trained on those explanations. To train EXAID-familiar we extracted adversarial negative using a FGSM attack with a noise level of = 0.1. // Train classifier for class i 17: end for As described, we used SHAP as an explainability model (a). The original implementation of SHAP runs on CPU and is therefore suitable for use only on a small number of samples, rather than on entire datasets. To enable us to run SHAP on a large number of examples we modified the original implementation to run on GPU. The new version is available in our Git repository. We compared EXAID with three recently-published adversarial detection baselines, and two new variants of these baselines. ANR . A method based on measuring the effect of quantization and smoothing of the image on the network classification, both are parameterized as a function of the image entropy. We used the implementation provided by the authors. Since ANR was not tested in the original paper on CIFAR-10 and SVHN as done here, we tuned the hyperparameters of their method using hyperopt . Mahanalobis . This approach models the distribution of activations in the hidden layers of the classifier, as obtained in response to natural (unperturbed) samples, using a Gaussian mixture model. Given a set of likelihood scores from the GMM, a classifier is trained to determine if a set of activations is obtained in response to an adversarial example or a natural one. That classifier is trained on adversarial examples. We used the implementation provided by the authors, and as the original paper, we trained the classifier with adversarial examples crafted by FGSM. Mahanalobis Unsupervised. We modified the method of to reach an attack-agnostic baseline as follows. Instead of training an attack-dependent discriminator on adversarial samples, we estimated the likelihood of a set of network activations as the product of likelihoods of all layers. LID . LID measures the characteristics of the region surrounding a reference example, and give it a likelihood score. This is done separately for each representation of the example, in the classifier's hidden layers. As in Mahalanobis, a classifier is trained to determine if a set of activations is obtained in response to an adversarial example or a natural one. We used the implementation from , and trained the classifier with adversarial examples crafted by FGSM. We test the detection models against oblivious adversaries, an attack scenario in which an attacker has full knowledge about the model (white box attack), but is not aware of the existence of the defense model. We believe this is a relevant scenario, since in the real world, most attackers will not have direct access to the attacked model and its defense. In this case, the attacker will be forced to use a black box attack. However, as showed, adversarial examples are transferable between models. Given transferability, attacking a black box model is not marginally harder than a white box. Because of that, we baseline our model against a white box attack as which is harder to detect. This is not the case when the model is defended, since shows that the transferability of adversarial examples works well between vanilla neural networks, but fail to transfer between defended neural networks. We believe that the magnitude of perturbation used by an attack is a major factor that determines the success of adversarial detection methods. There is still no clear protocol in the literature about comparing attacks and detections depending on this factor, and different reported experiments use different values. We therefore repeated all experiments for a wide range of noise levels and report performance across that wide range. We used three attack methods to test EXAID: One step gradient attack (FGSM) , Iterative projected gradient (PGD) and the Carlini and Wagner attack, which uses optimization to add as small as possible perturbation (C&W) (b). All attacks were implemented using Advertorch . Opposed to other defense methods benchmarks, we examined the effect of noise-level on a range of three orders of magnitude. The for all detection methods are shown in figure 2. EXAID significantly outperforms the other methods when the noise level is small (small perturbations), and with attack methods that use adaptive noise levels (C&W). Typically, the AUC is increased from 70% to over 90%. LID and Mahalanobis both perform well in high noise scenarios, and slightly outperform EXAID on SVHN in these scenarios. However, when the noise level decreases LID and Mahalanobis performance suffers drastically, while EXAID's remains high. Interestingly, our unsupervised variant of LID, performs at least as well, and sometimes better, than the original LID. This may be because LID was trained with FGSM samples and may deteriorate in cross-attack scenarios. These findings show the importance of benchmarking defense models against a wide range of noise levels. In this paper we proposed EXAID, a novel attack-detection approach, which uses model explainability to identify images whose explanations are inconsistent with the predicted class. Our method outperforms previous state-of-the-art methods, for three attack methods, and many noise-levels. We demonstrated that the attack noise level has a major impact on previous defense methods. We hope this will encourage the research community to evaluate future defense methods on a large range of noise-levels.

represent the tldr for finding the corresponding scientific article
------
Examples:

"We used a CVAE type model structure to learn to directly generate slates/whole pages for recommendation systems." == "The conventional approach to solving the recommendation problem greedily ranks individual document candidates by prediction scores. However, this method fails to optimize the slate as a whole, and hence, often struggles to capture biases caused by the page layout and document interdepedencies. The slate recommendation problem aims to directly find the optimally ordered subset of documents (i.e. slates) that best serve users’ interests. Solving this problem is hard due to the combinatorial explosion of document candidates and their display positions on the page. Therefore we propose a paradigm shift from the traditional viewpoint of solving a ranking problem to a direct slate generation framework. In this paper, we introduce List Conditional Variational Auto-Encoders (ListCVAE), which learn the joint distribution of documents on the slate conditioned on user responses, and directly generate full slates. Experiments on simulated and real-world data show that List-CVAE outperforms greedy ranking methods consistently on various scales of documents corpora. Recommender systems modeling is an important machine learning area in the IT industry, powering online advertisement, social networks and various content recommendation services BID0 ). In the context of document recommendation, its aim is to generate and display an ordered list of "documents" to users (called a "slate" in BID2 ; BID3), based on both user preferences and documents content. For large scale recommender systems, a common scalable approach at inference time is to first select a small subset of candidate documents S out of the entire document pool D. This step is called "candidate generation". Then a function approximator such as a neural network (e.g., a Multi-Layer Perceptron (MLP)) called the "ranking model" is used to predict probabilities of user engagements for each document in the small subset S and greedily generates a slate by sorting the top documents from S based on estimated prediction scores BID4. This two-step process is widely popular to solve large scale recommendation problems due to its scalability and fast inference at serving time. The candidate generation step can decrease the number of candidates from millions to hundreds or less, effectively dealing with scalability when faced with a large corpus of documents D. Since |S| is much smaller than |D|, the ranking model can be reasonably complicated without increasing latency. However, there are two main problems with this approach. First the candidate generation and the ranking models are not trained jointly, which can lead to having candidates in S that are not the highest scoring documents of the ranking model. Second and most importantly, the greedy ranking method suffers from numerous biases that come with the visual presentation of the slate and context in which documents are presented, both at training and serving time. For example, there exists positional biases caused by users paying more attention to prominent slate positions BID5, and contextual biases, due to interactions between documents presented together in the same slate, such as competition and complementarity, relative attractiveness, etc..In this paper, we propose a paradigm shift from the traditional viewpoint of solving a ranking problem to a direct slate generation framework. We consider a slate "optimal" when it maximizes some type of user engagement feedback, a typical desired scenario in recommender systems. For example, given a database of song tracks, the optimal slate can be an ordered list (in time or space) of k songs such that the user ideally likes every song in that list. Another example considers news articles, the optimal slate has k ordered articles such that every article is read by the user. In general, optimality can be defined as a desired user response vector on the slate and the proposed model should be agnostic to these problem-specific definitions. Solving the slate recommendation problem by direct slate generation differs from ranking in that first, the entire slate is used as a training example instead of single documents, preserving numerous biases encoded into the slate that might influence user responses. Secondly, it does not assume that more relevant documents should necessarily be put in earlier positions in the slate at serving time. Our model directly generates slates, taking into account all the relevant biases learned through training. In this paper, we apply Conditional Variational Auto-Encoders (CVAEs) BID7 BID8 to model the distributions of all documents in the same slate conditioned on the user response. All documents in a slate along with their positional, contextual biases are jointly encoded into the latent space, which is then sampled and combined with desired conditioning for direct slate generation, i.e. sampling from the learned conditional joint distribution. Therefore, the model first learns which slates give which type of responses and then directly generates similar slates given a desired response vector as the conditioning at inference time. We call our proposed model List-CVAE. The key contributions of our work are:1. To the best of our knowledge, this is the first model that provides a conditional generative modeling framework for slate recommendation by direct generation. It does not necessarily require a candidate generator at inference time and is flexible enough to work with any visual presentation of the slate as long as the ordering of display positions is fixed throughout training and inference times.2. To deal with the problem at scale, we introduce an architecture that uses pretrained document embeddings combined with a negatively downsampled k-head softmax layer within the List-CVAE model, where k is the slate size. The structure of this paper is the following. First we introduce related work using various CVAE-type models as well as other approaches to solve the slate generation problem. Next we introduce our List-CVAE modeling approach. The last part of the paper is devoted to experiments on both simulated and the real-world datasets.2 RELATED WORK Traditional matrix factorization techniques have been applied to recommender systems with success in modeling competitions such as the Netflix Prize BID10. Later research emerged on using autoencoders to improve on the of matrix factorization BID11 (CDAE, CDL). More recently several works use Boltzmann Machines BID13 and variants of VAE models in the Collaborative Filtering (CF) paradigm to model recommender systems BID14 BID15 BID16 ) (Collaborative VAE, JMVAE, CVAE-CF, JVAE-CF). See FIG0 for model structure comparisons. In this paper, unless specified otherwise, the user features and any context are routinely considered part of the conditioning variables (in Appendix A Personalization Test, we test List-CVAE generating personalized slates for different users). These models have primarily focused on modeling individual document or pairs of documents in the slate and applying greedy ordering at inference time. Our model is also using a VAE type structure and in particular, is closely related to the Joint Multimodel Variational Auto-Encoder (JMVAE) architecture FIG0 ). However, we use whole slates as input instead of single documents, and directly generate slates instead of using greedy ranking by prediction scores. Other relevant work from the Information Retrieval (IR) literature are listwise ranking methods BID17 BID18 BID19 BID20 BID21. These methods use listwise loss functions that take the contexts and positions of training examples into account. However, they eventually assign a prediction score for each document and greedily rank them at inference time. In the Reinforcement Learning (RL) literature, BID3 view the whole slates as actions and use a deterministic policy gradient update to learn a policy that generates these actions, given concatenated document features as input. Finally, the framework proposed by BID22 predicts user engagement for document and position pairs. It optimizes whole page layouts at inference time but may suffer from poor scalability due to the combinatorial explosion of all possible document position pairs. We formally define the slate recommendation problem as follows. Let D denote a corpus of documents and let k be the slate size. Then let r = (r 1, . . ., r k) be the user response vector, where r i ∈ R is the user's response on document d i. For example, if the problem is to maximize the number of clicks on a slate, then let r i ∈ {0, 1} denote whether the document d i is clicked, and thus an optimal slate DISPLAYFORM0 Variational Auto-Encoders (VAEs) are latent-variable models that define a joint density P θ (x, z) between observed variables x and latent variables z parametrized by a vector θ. Training such models requires marginalizing the latent variables in order to maximize the data likelihood P θ (x) = P θ (x, z)dz. Since we cannot solve this marginalization explicitly, we resort to a variational approximation. For this, a variational posterior density Q φ (z|x) parametrized by a vector φ is introduced and we optimize the variational Evidence Lower-Bound (ELBO) on the data loglikelihood: DISPLAYFORM0 DISPLAYFORM1 where KL is the Kullback-Leibler divergence and where P θ (z) is a prior distribution over latent variables. In a Conditional VAE (CVAE) we extend the distributions P θ (x, z) and Q φ (z|x) to also depend on an external condition c. The corresponding distributions are indicated by P θ (x, z|c) and Q φ (z|x, c). Taking the conditioning c into account, we can write the variational loss to minimize as DISPLAYFORM2 We assume that the slates s = (d 1, d 2, . . . d k) and the user response vectors r are jointly drawn from a distribution P D k ×R k. In this paper, we use a CVAE to model the joint distribution of all DISPLAYFORM0 DISPLAYFORM1 is the conditioning vector, where r = (r 1, r 2, . . ., r k) is the user responses on the slate s. The concatenation of s and c makes the input vector to the encoder. The latent variable z ∈ R m has a learned prior distribution N (µ 0, σ 0). The raw output from the decoder are k vectors x 1, x 2..., x k, each of which is mapped to a real document through taking the dot product with the matrix Φ containing all document embeddings. Thus produced k vectors of logits are then passed to the negatively downsampled k-head softmax operation. At inference time, c is the ideal condition whose concatenation with sampled z is the input to the decoder. documents in the slate conditioned on the user responses r, i.e. P(d 1, d 2, . . . d k |r). At inference time, the List-CVAE model attempts to generate an optimal slate by conditioning on the ideal user response r.As we explained in Section 1, "optimality" of a slate depends on the task. With that in mind, we define the mapping Φ: R k → C. It transforms a user response vector r into a vector in the conditioning space C that encodes the user engagement metric we wish to optimize for. For instance, if we want to maximize clicks on the slate, we can use the binary click response vectors and set the conditioning to c = Φ(r):= k i=0 r i. Then at inference time, the corresponding ideal user response r would be (1, 1, . . ., 1), and correspondingly the ideal conditioning would be c = Φ(r) = k i=0 1 = k. As usual with CVAEs, the decoder models a distribution P θ (s|z, c) that, conditioned on z, is easy to represent. In our case, P θ (s|z, c) models an independent probability for each document on the slate, represented by a softmax distribution. Note that the documents are only independent to each other conditional on z. In fact, the marginalized posterior P θ (s|c) = z P θ (s|z, c)P θ (z|c)dz can be arbitrarily complex. When the encoder encodes the input slate s into the latent space, it learns the joint distribution of the k documents in a fixed order, and thus also encodes any contextual, positional biases between documents and their respective positions into the latent variable z. The decoder learns these biases through reconstruction of the input slates from latent variables z with conditions. At inference time, the decoder reproduces the input slate distribution from the latent variable z with the ideal conditioning, taking into account all the biases learned during training time. Figure 3: Predictive prior distribution of the latent variable z in R 2, conditioned on ideal user response c = (1, 1, . . ., 1). The color map corresponds to the expected total responses of the corresponding slates. Plots are generated from the simulation experiment with 1000 documents and slate size 10.To shed light onto what is encoded in the latent space, we simplify the prior distribution of z to be a fixed Gaussian distribution N (0, I) in R 2. We train List-CVAE and plot the predictive prior z. As training evolves, generated output slates with low total responses are pushed towards the edge of the latent space while high response slates cluster towards a growing center area (Figure 3). Therefore after training, if we sample z from its prior distribution N (0, I) and generate the corresponding output slates, they are more likely to have high total responses. Since the number of documents in D can be large, we first embed the documents into a low dimensional space. Let Ψ: D → S q−1 be that normalized embedding where S q−1 denotes the unit sphere in R q. Ψ can easily be pretrained using a standard supervised model that predicts user responses from documents or through a standard auto-encoder technique. For the i-th document in the slate, our model produces a vector x i in R q that is then matched to each document from D via a dot-product. This operation produces k vectors of logits for k softmaxes, i.e. the k-head softmax. At training time, for large document corpora D, we uniformly randomly downsample negative documents and compute only a small subset of the logits for every training example, therefore efficiently scaling the nearest neighbor search to millions of documents with minimal model quality loss. We train this model as a CVAE by minimizing the sum of the reconstruction loss and the KLdivergence term: DISPLAYFORM2 where β is a function of the training step BID23.During inference, output slates are generated by first sampling z from the conditionally learned prior distribution N (µ, σ), concatenating with the ideal condition c = Φ(r), and passed into the decoder, generating (x 1, . . ., x k) from the learned P θ (s|z, c), and finally taking arg max over the dot-products with the full embedding matrix independently for each position i = 1,..., k. for i = 1,..., k, where B represents the Bernoulli distribution. During training, all models see uniformly randomly generated slates s ∼ U({1, n} k ) and their generated responses r. During inference time, we generate slates s by conditioning on c = (1, . . ., 1). We do not require document de-duplication since repetition may be desired in certain applications (e.g. generating temporal slates in an online advertisement session). Moreover List-CVAE should learn to produce the optimal slates whether those slates contain duplication or not from learning the training data distribution. Evaluation: For evaluation, we cannot use offline ranking evaluation metrics such as Normalized Discounted Cumulative Gain (NDCG) BID24, Mean Average Precision (MAP) or Inverse Propensity Score (IPS) BID26, etc. These metrics either require prediction scores for individual documents or assume that more relevant documents should appear in earlier ranking positions, unfairly favoring greedy ranking methods. Moreover, we find it limiting to use various diversity metrics since it is not always the case that a higher diversity-inclusive score gives better slates measured by user's total responses. Even though these metrics may be more transparent, they do not measure our end goal, which is to maximize the expected number of total responses on the generated slates. Instead, we evaluate the expected number of clicks over the distribution of generated slates and over the distribution of clicks on each document: DISPLAYFORM0 In practice, we distill the simulated environment of Eq. 5 using the cross-entropy loss onto a neural network model that officiates as our new simulation environment. The model consists of an embedding layer, which encodes documents into 8-dimensional embeddings. It then concatenates the embeddings of all the documents that form a slate and follows this concatenation with two hidden layers and a final softmax layer that predicts the slate response amongst the 2 k possible responses. Thus we call it the "response model". We use the response model to predict user responses on 100,000 sampled output slates for evaluation purposes. This allows us to accurately evaluate our output slates by List-CVAE and all other baseline models. Our experiments compare List-CVAE with several greedy ranking baselines that are often deployed in industry productions, namely Greedy MLP, Pairwise MLP, Position MLP and Greedy Long Short-Term Memory (LSTM) models. In addition to the greedy baselines, we also compare against auto-regressive (AR) versions of Position MLP and LSTM, as well as randomly-selected slates from the training set as a sanity check. List-CVAE generates slates s = arg max s∈{1,...,n} k P θ (s|z, c). The encoder and decoder of List-CVAE, as well as all the MLP-type models consist of two fully-connected neural network layers of the same size. Greedy MLP trains on (d i, r i) pairs and outputs the greedy slate consisting of the top k highestP (r|d) scoring documents. Pairwise MLP is an MLP model with a pairwise ranking loss DISPLAYFORM0 where L x is the cross entropy loss and (x 1, x 2) are pairs of documents randomly selected with different user responses from the same slate. We sweep on hyperparameters α and η in addition to the shared MLP model structure sweep. Position MLP uses position in the slate as a feature during training time and simply sets it to 0 for fast performance at inference time. AR Position MLP is identical to Position MLP with the exception that the position feature is set to each corresponding slate position at inference time (as such it takes into account position biases). Greedy LSTM is an LSTM model with fully-connect layers before and after the recurrent middle layers. We tune the hyperparameters corresponding to the number of layers and their respective widths. We use sequences of documents that form slates as the input at training time, and use single examples as inputs with sequence length 1 at inference time, which is similar to scoring documents as if they are in the first position of a slate of size 1. Then we greedily rank the documents based on their prediction scores. AR LSTM is identical to Greedy LSTM during training. During inference, however, it selects documents sequentially by first selecting the best document for position 1, then unrolling the LSTM for 2 steps to select the best document for position 2, and so on. This way it takes into account the context of all previous documents and their positions. Random selects slates uniformly randomly from the training set. Small-scale experiment (n = 100, 1000, k = 10): 0 1000 2000 3000 4000 5000Step FORMULA1 Step FORMULA2 We use the trained document embeddings from the response model for List-CVAE and all the baseline models. For List-CVAE, we also use trained priors P θ (z |c) = N (µ, σ) where µ, σ = f prior (c) and f prior is modeled by a small MLP. Additionally, since we found little difference between different hyperparameters, we fixed the width of all hidden layers to 128, the learning rate to 10 DISPLAYFORM1 and the number of latent dimensions to 16. For all other baseline models, we sweep on the learning rates, model structures and any model specific hyperparameters such as α, η for Position MLP and the forget bias for the LSTM model. FIG5 show the performance comparison when the number of documents n = 100, 1000 and slate size to k = 10. While List-CVAE is not quite capable of reaching a perfect performance of 10 clicks (which is probably even above the optimal upper bound), it easily outperforms all other ranking baselines after only a few training steps. Appendix A includes an additional personalization test. Due to a lack of publicly available large scale slate datasets, we use the data provided by the RecSys 2015 YOOCHOOSE Challenge BID27. This dataset consists of 9.2M user purchase sessions around 53K products. Each user session contains an ordered list of products on which the user clicked, and whether they decided to buy them. The List-CVAE model can be used on slates with temporal ordering. Thus we form slates of size 5 by taking consecutive clicked products. We then build user responses from whether the user bought them. We remove a portion of slates with no positive responses such that after removal they only account for 50% of the total number of slates. After filtering out products that are rarely bought, we get 375K slates of size 5 and a corpus of 10,000 candidate documents. FIG9 shows the user response distribution of the training data. Notice that in the response vector, 0 denotes a click without purchase and 1 denotes a purchase. For example, means the user clicked on all five products but bought only the first and the last products. Medium-scale experiment (n = 10, 000, k = 5):Similarly to the previous section, we train a two-layer response model that officiates as a new semisynthetic simulation environment. We use the same hyperparameters used previously. FIG9 shows that List-CVAE outperforms all other baseline models within 500 training steps, which corresponds to having seen less than 10 −11 % of all possible slates. Large-scale experiment (n = 1 million, 2 millions, k = 5):We synthesize 1,990k documents by adding independent Gaussian noise N (0, 10 −2 · I) to the original 10k documents and label the synthetic documents by predicted responses from the response model. Thus the new pool of candidate documents consists of 10k original documents and 1,990k synthetic ones, totaling 2 million documents. To match each of the k decoder outputs (x 1, x 2, . . ., x k) with Step 0 Step 0 Step 0 real documents, we uniformly randomly downsample the negative document examples keeping in total only 1000 logits (the dot product outputs in the decoder) during training. At inference time, we pick the argmax for each of k dot products with the full embedding matrix without sampling. This technique speeds up the total training and inference time for 2 million documents to merely 4 minutes on 1 GPU for both the response model (with 40k training steps) and List-CVAE (with 5k training steps). We ran 2 experiments with 1 million and 2 millions document respectively. From the shown in FIG9 and 5d, List-CVAE steadily outperforms all other baselines again. The greatly increased number of training examples helped List-CVAE really learn all the interactions between documents and their respective positional biases. The ing slates were able to receive close to 5 purchases on average due to the limited complexity provided by the response model. In practice, we may not have any close-to-optimal slates in the training data. Hence it is crucial that List-CVAE is able to generalize to unseen optimal conditions. To test its generalization capacity, we use the medium-scale experiment setup on RecSys 2015 dataset and eliminate from the training data all slates where the total user response exceeds some ratio h of the slate size k, i.e. k i=1 r i > hk for h = 80%, 60%, 40%, 20%. FIG10 shows test on increasingly difficult training sets from which to infer on the optimal slates. Without seeing any optimal slates FIG10 ) or slates with 4 or 5 total purchases FIG10 ), List-CVAE can still produce close to optimal slates. Even training on slates with only 0, 1 or 2 total purchases (h = 40%), List-CVAE still surpasses the performance of all greedy baselines within 1000 steps FIG10. Thus demonstrating the strong generalization power of the model. List-CVAE cannot learn much about the 0 1000 2000 3000 4000 5000Step 0 interactions between documents given only 0 or 1 total purchase per slate FIG10 ), whereas the MLP-type models learn purchase probabilities of single documents in the same way as in slates with higher responses. Although evaluation of our model requires choosing the ideal conditioning c at or near the edge of the support of P (c), we can always compromise generalization versus performance by controlling c in practice. Moreover, interactions between documents are governed by similar mechanisms whether they are from the optimal or sub-optimal slates. As the experiment indicate, List-CVAE can learn these mechanisms from sub-optimal slates and generalize to optimal slates. The List-CVAE model moves away from the conventional greedy ranking paradigm, and provides the first conditional generative modeling framework that approaches slate recommendation problem using direct slate generation. By modeling the conditional probability distribution of documents in a slate directly, this approach not only automatically picks up the positional and contextual biases between documents at both training and inference time, but also gracefully avoids the problem of combinatorial explosion of possible slates when the candidate set is large. The framework is flexible and can incorporate different types of conditional generative models. In this paper we showed its superior performance over popular greedy and auto-regressive baseline models with a conditional VAE model. In addition, the List-CVAE model has good scalability. We designed an architecture that uses pretrained document embeddings combined with a negatively downsampled k-head softmax layer that greatly speeds up the training, scaling easily to millions of documents. This test complements the small-scale experiment. To the 100 documents with slate size 10, we add user features into the conditioning c, by adding a set U of 50 different users to the simulation engine (|U| = 50, n = 100, k = 10), permuting the innate attractiveness of documents and their interactions matrix W by a user-specific function π u. Let be the response of the user u on the document d i. During training, the condition c is a concatenation of 16 dimensional user embeddings Θ(u) obtained from the response model, and responses r. At inference time, the model conditions on c = (r, Θ(u)) for each randomly generated test user u. We sweep over hidden layers of 512 or 1024 units in List-CVAE, and all baseline MLP structures. FIG12 show that slates generated by List-CVAE have on average higher clicks than those produced by the greedy baseline models although its convergence took longer to reach than in the small-scale experiment."

A novel adversarial detection approach, which uses explainability methods to identify images whose explanations are inconsistent with the predicted class.

Represent this text to find a much much longer article about it with say hundreds-thousands of words


Question: represent the text to find a very short summary (1-2 sentences max)

Question: We describe a simple and general neural network weight compression approach, in which the network parameters (weights and biases) are represented in a “latent” space, amounting to a reparameterization. This space is equipped with a learned probability model, which is used to impose an entropy penalty on the parameter representation during training, and to compress the representation using a simple arithmetic coder after training. Classification accuracy and model compressibility is maximized jointly, with the bitrate--accuracy trade-off specified by a hyperparameter. We evaluate the method on the MNIST, CIFAR-10 and ImageNet classification benchmarks using six distinct model architectures. Our show that state-of-the-art model compression can be achieved in a scalable and general way without requiring complex procedures such as multi-stage training. Artificial neural networks (ANNs) have proven to be highly successful on a variety of tasks, and as a , there is an increasing interest in their practical deployment. However, ANN parameters tend to require a large amount of space compared to manually designed algorithms. This can be problematic, for instance, when deploying models onto devices over the air, where the bottleneck is often network speed, or onto devices holding many stored models, with only few used at a time. To make these models more practical, several authors have proposed to compress model parameters (; ; ;). While other desiderata often exist, such as minimizing the number of layers or filters of the network, we focus here simply on model compression algorithms that 1. minimize compressed size while maintaining an acceptable classification accuracy, 2. are conceptually simple and easy to implement, and 3. can be scaled easily to large models. Classic data compression in a Shannon sense requires discrete-valued data (i.e., the data can only take on a countable number of states) and a probability model on that data known to both sender and receiver. Practical compression algorithms are often lossy, and consist of two steps. First, the data is subjected to (re-)quantization. Then, a Shannon-style entropy coding method such as arithmetic coding is applied to the discrete values, bringing them into a binary representation which can be easily stored or transmitted. Shannon's source coding theorem establishes the entropy of the discrete representation as a lower bound on the average length of this binary sequence (the bit rate), and arithmetic coding achieves this bound asymptotically. Thus, entropy is an excellent proxy for the expected model size. The type of quantization scheme affects both the fidelity of the representation (in this case, the precision of the model parameters, which in turn affects the prediction accuracy) as well as the bit rate, since a reduced number of states coincides with reduced entropy. ANN parameters are typically represented as floating point numbers. While these technically have a finite (but large) number of states, the best in terms of both accuracy and bit rate are typically achieved for a significantly reduced number of states. Existing approaches to model compression often acknowledge this by quantizing each individual linear filter coefficient in an ANN to a small number of pre-determined values (; ; F.). This is known as scalar quantization (SQ). Other methods explore vector quantization (VQ), which is closely related to k-means clustering, in which each vector of filter coefficients is quantized jointly (Chen, J. ;). This is equivalent to enumerating a finite set of representers Figure 1: Visualization of representers in scalar quantization vs. reparameterized quantization. The axes represent two different model parameters (e.g., linear filter coefficients). Small black dots are samples of the model parameters, red and blue discs are the representers. Left: in scalar quantization, the representers must be given by a Kronecker product of scalar representers along the cardinal axes, even though the distribution of samples may be skewed. Right: in reparameterized scalar quantization, the representers are still given by a Kronecker product, but in a transformed (here, rotated) space. This allows a better adaptation of the representers to the parameter distribution. (representable vectors), while in SQ the set of representers is given by the Kronecker product of representable scalar elements. VQ is much more general than SQ, in the sense that representers can be placed arbitrarily: if the set of useful filter vectors all live in a subset of the entire space, there is no benefit in having representers outside of that subset, which may be unavoidable with SQ (Figure 1, left). Thus, VQ has the potential to yield better , but it also suffers from the "curse of dimensionality": the number of necessary states grows exponentially with the number of dimensions, making it computationally infeasible to perform VQ for much more than a handful of dimensions. One of the key insights leading to this paper is that the strengths of SQ and VQ can be combined by representing the data in a "latent" space. This space can be an arbitrary rescaling, rotation, or otherwise warping of the original data space. SQ in this space, while making quantization computationally feasible, can provide substantially more flexibility in the choice of representers compared to the SQ in the data space (Figure 1, right). This is in analogy to recent image compression methods based on autoencoders (Ballé, ;). The contribution of this paper is two-fold. First, we propose a novel end-to-end trainable model compression method that uses scalar quantization and entropy penalization in a reparameterized space of model parameters. The reparameterization allows us to use efficient SQ, while achieving flexibility in representing the model parameters. Second, we provide state-of-the-art on a variety of network architectures on several datasets. This demonstrates that more complicated strategies involving pretraining, multi-stage training, sparsification, adaptive coding, etc., as employed by many previous methods, are not necessary to achieve good performance. Our method scales to modern large image datasets and neural network architectures such as ResNet-50 on ImageNet. We consider the classification setup, where we are given a dataset D = {(x 1, y 1),...(x N, y N)} consisting of pairs of examples x i and corresponding labels y i. We wish to minimize the expected negative log-likelihood on D, or cross-entropy classification loss, over the set of model parameters Θ: where p(y | x; Θ) is the likelihood our model assigns to a dataset sample (x, y). The likelihood function is implemented using an ANN with parameters where W k and b k denote the weight (including convolutional) and bias terms at layer k, respectively. Compressing the model amounts to compressing each parameter in the set Θ. Instead of compressing each parameter directly, we compress reparameterized forms of them. To be precise, we introduce the reparameterizations Figure 2: Classifier architecture. The Φ tensors (annotated with a tilde) are stored in their compressed form. During inference, they are read from storage, uncompressed, and transformed via f into Θ, the usual parameters of a convolutional or dense layer (denoted without a tilde). The internals of f conv and f dense in our experiments for layer k, annotated with the dimensionalities. In f conv, H, W, I, O refer to the convolutional height, width, input channel, output channel, respectively. For f dense, I and O refer to the number of input and output activations. For f conv, we use an affine transform, while for f dense we use a scalar shift and scale, whose parameters are captured in Ψ. Note that in both cases, the number of parameters of f itself (labeled as ψ) is significantly smaller than the size of the model parameters it decodes. f bias such that We can think of each parameter decoder f as a mapping from reparameterization space to parameter space. For ease of notation, we write F = {f conv, f dense, f bias} and Θ = F(Φ). The parameter decoders themselves may have learnable parameters, which we denote Ψ. Our method is visually summarized in figures 2 and 3. A central component of our approach is partitioning the set of model parameters into groups. For the purpose of creating a model compression method, we interpret entire groups of model parameters as samples from the same learned distribution. We define a fully factorized distribution q(Φ) = φ∈Φ q φ (φ), and introduce parameter sharing within the factors q φ of the distribution that correspond to the same group, as well as within the corresponding decoders. These group assignments are fixed a priori. For instance, in figure 2, W 1 and W 2 can be assumed to be samples of the same distribution, that is q W 1 (·) = q W 2 (·). To be consistent, we also use the same parameter decoder f conv to decode them. Further, each of the reparameterizations φ is defined as a rank-2 tensor (a matrix), where each row corresponds to a "sample" from the learned distribution. The operations in f apply the same transformation to each row (figure 3). As an example, in f conv, each spatial H × W matrix of filter coefficients is assumed to be a sample from the same distribution. We describe how this aids in compression in the following section. Our method can be applied analogously to various model partitionings. In fact, in our experiments, we vary the size of the groups, i.e., the number of parameters assumed i.i.d., depending on the total number of parameters of the model (Θ). The size of the groups parameterizes a trade-off between compressibility and overhead: if groups consisted of just one scalar parameter each, compressibility would be maximal, since q would degenerate (i.e., would capture the value of the parameter with certainty). However, the overhead would be maximal, since F and q would have a large number of parameters that would need to be included in the model size (defeating the purpose of compression). On the other hand, encoding all parameters of the model with one and the same decoder and scalar distribution would minimize overhead, but may be overly restrictive by failing to capture distributional differences amongst all the parameters, and hence lead to suboptimal compressibility. We describe the group structure of each network that we use in more detail in the experiments section. In order to apply a Shannon-style entropy coder efficiently to the reparameterizations Φ, we need a discrete alphabet of representers and associated probabilities for each representer. Rather than handling an expressive set of representers, as in VQ, we choose to fix them to the integers, and achieve expressivity via the parameter decoders F instead. Each φ ∈ Z d× is a matrix interpreted as consisting of d samples from a discrete probability distribution producing vectors of dimension. We fit a factorized probability model to each column i of φ, using different probability models q i for each corresponding parameter decoder (the form of q i is described in the next section). Fitting of probability models is typically done by minimizing the negative log-likelihood. Assuming φ follows the distribution q, Shannon's source coding theorem states that the minimal length of a bit sequence encoding φ is the selfinformation of φ under q: which is identical to Shannon cross entropy up to an expectation operator, and identical to the negative log likelihood up to a constant factor. By minimizing I over q and φ during training, we thus achieve two goals: 1) we fit q to the model parameters in a maximum likelihood sense, and 2) we directly optimize the parameters for compressibility. After training, we design an arithmetic code for q, and use it to compress the model parameters. This method incurs only a small overhead over the theoretical bound due to the finite length of the bit sequence (arithmetic coding is asymptotically optimal). Practically, the overhead amounts to less than 1% of the size of the bit sequence; thus, self-information is an excellent proxy for model size. Further overhead from including a description of Ψ, the parameters of the parameter decoders, as well as of q itself (in the form of a table) in the model size. However, these can be considered constant and small compared to the total model size, and thus do not need to be explicitly optimized for. The overall loss function is simply the additive combination of the original cross-entropy classification loss under reparameterization with the self-information of all reparameterizations: We refer to the second term (excluding the constant λ) as the rate loss. By varying λ across different experiments, we can explore the Pareto frontier of compressed model size vs. model accuracy. To compare our method to other work, we varied λ such that our method produced similar accuracy, and then compared the ing model size. Since Φ is discrete-valued, we need to make some further approximations in order to optimize L over it using stochastic gradient descent. To get around this, we maintain continuous surrogatesΦ. For optimizing the classification loss, we use the "straight-through" gradient estimator , which provides a biased gradient estimate but has shown good in practice. This consists of rounding the continuous surrogate to the nearest integer during training, and ignoring the rounding for purposes of backpropagation. After training, we only keep the discretized values. In order to obtain good estimates for both the rate term and its gradient during training, we adopt a relaxation approach previously described in (Ballé,); the code is provided as an open source library 1. In a nutshell, the method replaces the probability mass functions q i with a set of non-parametric continuous density functions, which are based on small ANNs. These density models are fitted toφ j,i + n j,i, where n j,i ∼ U(− 1 2, 1 2) is i.i.d. uniformly distributed additive noise. This turns out to work well in practice, because the negative log likelihood of these noise-affected variates under the continuous densities approximates the self-information I: whereq i denote the density functions. Once the density models are trained, the values of the probability mass functions modeling φ are derived from the substitutesq i and stored in a table, which is included in the model description. The parameters ofq i are no longer needed after training. For our MNIST and CIFAR-10 experiments, we evaluate our method by applying it to four distinct image classification networks: LeNet300-100 and LeNet-5-Caffe 2 on MNIST , and VGG-16 3 and ResNet-20 (b;) with width multiplier 4 (ResNet-20-4) on CIFAR-10 . For our ImageNet experiments, we evaluate our method on the ResNet-18 and ResNet-50 (a) networks. We train all our models from scratch and compare them with recent state-of-the-art methods by quoting performance from their respective papers. Compared to many previous approaches, we do not initialize the network with pre-trained or pre-sparsified weights. We found it useful to use two separate optimizers: one to optimize the variables of the probability model and one to optimize the variables of the network. The optimizer for the probability model is always Adam with a learning rate of 0.0001. We chose to always use Adam because the parameter updates used by Adam are independent of any scaling of the objective (when its hyper-parameter is sufficiently small). In our method, the probability model variables only get gradients from the entropy loss which is scaled by the rate penalty λ. Adam normalizes out this scale and makes the learning rate of the probability model independent of λ and of other hyperparameters such as the model partitioning. We apply our method to two LeNet variants: LeNet300-100 and LeNet5-Caffe and report in Table 1. We train the networks using Adam with a constant learning rate of 0.001 for 200,000 iterations. To remedy some of the training noise from quantization, we maintain an exponential moving average (EMA) of the weights and evaluate using those. Note that this does not affect the quantization, as quantization is performed after the EMA variables are restored. LeNet300-100 consists of 3 fully connected layers. We partitioned this network into three parameter groups: one for the first two fully connected layers, one for the classifier layer, and one for biases. LeNet5-Caffe consists of two 5×5 convolutional layers followed by two fully connected layers, with max pooling following each convolutional layer. We partitioned this network into four parameter groups: One for both of the convolutional layers, one for the penultimate fully connected layer, one for the final classifier layer, and one for the biases. As evident from Table 1, for the larger LeNet300-100 model, our method outperforms all the baselines while maintaining a comparable error rate. For the smaller LeNet5-Caffe model, our method is second only to Minimal Random Code Learning . Note that in both of the MNIST models, the number of probability distributions = 1 in every parameter group, including in the convolutional layers. To be precise, the W k for the convolutional weights W k will be H · W · I · O × 1. We found that this gives a better trade-off, since the model is small to begin with, and having = 5 · 5 = 25 scalar probability models for 5 × 5 convolutional layers would have too much overhead. For both of the MNIST models, we found that letting each subcomponent of F be a simple dimension-wise scalar affine transform (similar to f dense in figure 3), was sufficient. Since each φ is quantized to integers, having a flexible scale and shift leads to flexible SQ, similar to in . Due to the small size of the networks, more complex transformation functions lead to too much overhead. We apply our method to VGG-16 and ResNet-20-4 (b;) and report the in Table 1. For both VGG-16 and ResNet-20-4, we use momentum of 0.9 with an initial learning rate of 0.1, and decay by 0.2 at iterations 256,000, 384,000, and 448,000 for a total of 512,000 iterations. This learning rate schedule was fixed from the beginning and was not tuned in any way other than verifying that our models' training loss had converged. VGG-16 consists of 13 convolutional layers of size 3 × 3 followed by 3 fully connected layers. We split this network into four parameter groups: one for all convolutional layers and one each all fully connected layers. We do not compress biases. We found that our biases in float32 format add up to about 20 KB, and we add that to our reported numbers. ResNet-20-4 consists of 3 ResNet groups with 3 residual blocks each. There is also an initial convolution layer and a final fully connected classification layer. We partition this network into two parameter groups: one for all convolutional layers and one for the final classification layer. We also do not compress biases but include them in our ; they add up to about 11 KB. For VGG-16 and ResNet-20-4 convolutions, = O × I = 9; f conv and f dense are exactly as pictured in figure 3. To speed up training, we fixed ψ W. We found that the inverse real-valued discrete Fourier transform (DFT) performs much better than SQ, or any random orthogonal matrix (Figure 4). From the error vs. rate plots, the benefit of reparameterization in the high compression regime is evident. VGG-16 and ResNet-20-4 both contain batch normalization layers that include a moving average for the mean and variance. Following , we do not include the moving averages in our reported numbers. We do, however, include the batch normalization bias term β and let it function as the bias for each layer (γ is set to a constant 1). For the ImageNet dataset , we reproduce the training setup and hyperparameters from He et al. (2016a). All 3x3 convolutional layers belong to a single parameter group, similar to our CIFAR experiments, 1x1 convolutional layers to a single group (applicable to, and all the remaining layers in their own groups. This gives a total of 4 parameter groups for ResNet-50 and 3 groups for ResNet-18. Analogously to the CIFAR experiments, we compare SQ to using random orthogonal or DFT matrices for reparameterizing the convolution kernels (figure 4a). Existing model compression methods are typically built on a combination of pruning, quantization, or coding. Pruning involves sparsifying the network either by removing individual parameters or higher level structures such as convolutional filters, layers, activations, etc. Various strategies for pruning weights include looking at the Hessian or just their p norm . focus on pruning individual units, and H. prunes convolutional filters. and 18.2 KB (58x) 1.8% Bayesian Compression (GHS) 18.0 KB (59x) 2.0% Sparse Variational Dropout 9.38 KB (113x) 1.8% Our Method (SQ) 8.56 KB (124x) 1.9% LeNet5-Caffe (MNIST) Uncompressed 1.72 MB 0.7% Sparse Variational Dropout 4.71 KB (365x) 1.0% Bayesian Compression (GHS) 2.23 KB (771x) 1.0% Minimal Random Code Learning 1.52 KB (1110x) 1.0% Our Method (SQ) 2.84 KB (606x) 0.9% Uncompressed 60 MB 6.6% Bayesian Compression 525 KB (116x) 9.2% DeepCABAC 960 KB (62.5x) 9.0% Minimal Random Code Learning 417 KB (159x) 6.6% Minimal Random Code Learning 168 KB (452x) 10.0% Our Method (DFT) 101 KB (590x) 10.0% ResNet-20-4 (CIFAR-10) 6.46 MB (16x) 26.0% DeepCABAC 6.06 MB (17x to in our compression experiments, also prune parts of the network. describes a dimensionality reduction technique specialized for CNN architectures. Pruning is a simple approach to reduce memory requirements as well as computational complexity, but doesn't inherently tackle the problem of efficiently representing the parameters that are left. Here, we primarily focus on the latter: given a model architecture and a task, we're interested in finding a set of parameters which can be described in a compact form and yield good prediction accuracy. Our work is largely orthogonal to the pruning literature, and could be combined if reducing the number of units is desired. Quantization involves restricting the parameters to a small set of unique values. There is work in binarizing or ternarizing networks (; F. ;) via either straight-through gradient approximation or stochastic rounding . introduced a new differentiable quantization procedure that relaxes quantization. We use the straight-through heuristic, but could possibly use other stochastic approaches to improve our methods. While most of these works focus on uniform quantization, also extend to non-uniform quantization, which our generalized transformation function amounts to. and share weights and quantize by clustering, Chen, J. randomly enforce weight sharing, and thus effectively perform VQ with a pre-determined assignment of parameters to representers. Other works also make the observation that representing weights in the frequency domain helps compression; Chen, J. T. randomly enforce weight sharing in the frequency domain and use K-means clustering in the frequency domain. Coding (entropy coding, or Shannon-style compression) methods produce a bit sequence that can allow convenient storage or transmission of a trained model. This generally involves quantization as a first step, followed by methods such as Huffman coding , arithmetic coding , etc. Entropy coding methods exploit a known probabilistic structure of the data to produce optimized binary sequences whose length ideally closely approximates the cross entropy of the data under the probability model. In many cases, authors represent the quantized values directly as binary numbers with few digits (; F. ;), which effectively leaves the probability distribution over the values unexploited for minimizing model size; others do exploit it . formulate model compression with an entropy constraint, but use (non-reparameterized) scalar quantization. Their model significantly underperforms all the state-of-the-art models that we compare with (Table 1). Some recent work has claimed improved compression performance by skipping quantization altogether . Our work focuses on coding with quantization. defined their method using a four-stage training process: 1. training the original network, 2. pruning and re-training, 3. quantization and re-training, and 4. entropy coding. This approach has influenced many follow-up publications. In the same vein, many current high-performing methods have significant complexity in implementation or require a multi-stage training process. requires several stages of training and retraining while keeping parts of the network fixed. require pre-sparsification of the network, which is computationally expensive, and use a more complex (context-adaptive) variant of arithmetic coding which may be affected by MPEG patents. These complexities can prevent methods from scaling to larger architectures or decrease their practical usability. In contrast, our method requires only a single training stage followed by a royalty-free version of arithmetic coding. In addition, we commit to releasing the source code of our method for easy reproducibility (upon publication). Our method has parallels to recent work in learned image compression (Ballé, ;) that uses end-to-end trained deep models for significant performance improvements in lossy image compression. These models operate in an autoencoder framework, where scalar quantization is applied in the latent space. Our method can be viewed as having just a decoder that is used to transform the latent representation into the model parameters, but no encoder. We describe a simple model compression method built on two ingredients: joint (i.e., end-to-end) optimization of compressibility and classification performance in only a single training stage, and reparameterization of model parameters, which increases the flexibility of the representation over scalar quantization, and is applicable to arbitrary network architectures. We demonstrate that stateof-the-art model compression performance can be achieved with this simple framework, outperforming methods that rely on complex, multi-stage training procedures. Due to its simplicity, the approach is particularly suitable for larger models, such as VGG and especially ResNets. In future work, we may consider the potential benefits of even more flexible (deeper) parameter decoders.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

An end-to-end trainable model compression method optimizing accuracy jointly with the expected model size.

We develop VAEs where the encoder takes a model parameter vector as input, so we can do rapid inference for many models