Question: Represent the article for finding a single-sentence TLDR

Question: One of the most prevalent symptoms among the elderly population, dementia, can be detected by classifiers trained on linguistic features extracted from narrative transcripts. However, these linguistic features are impacted in a similar but different fashion by the normal aging process. Aging is therefore a confounding factor, whose effects have been hard for machine learning classifiers to isolate. In this paper, we show that deep neural network (DNN) classifiers can infer ages from linguistic features, which is an entanglement that could lead to unfairness across age groups. We show this problem is caused by undesired activations of v-structures in causality diagrams, and it could be addressed with fair representation learning. We build neural network classifiers that learn low-dimensional representations reflecting the impacts of dementia yet discarding the effects of age. To evaluate these classifiers, we specify a model-agnostic score $\Delta_{eo}^{(N)}$ measuring how classifier are disentangled from age. Our best models outperform baseline neural network classifiers in disentanglement, while compromising accuracy by as little as 2.56\% and 2.25\% on DementiaBank and the Famous People dataset respectively. One in three seniors die of Alzheimer's and other types of dementia in the United States . Although its causes are not yet fully understood, dementia impacts people's cognitive abilities in a detectable manner. This includes different syntactic distributions in narrative descriptions BID28, more pausing BID29, higher levels of difficulty in recalling stories BID21, and impaired memory generally BID20. Fortunately, linguistic features can be used to train classifiers to detect various cognitive impairments. For example, BID8 detected primary progressive aphasia with up to 100% accuracy, and classified subtypes of primary progressive aphasia with up to 79% accuracy on a set of 40 participants using lexical-syntactic and acoustic features. BID7 classified dementia from control participants with 82% accuracy on narrative speech. However, dementia is not the only factor causing such detectable changes in linguistic features of speech. Aging also impairs cognitive abilities BID11, but in subtly different ways from dementia. For example, aging inhibits fluid cognitive abilities (e.g., cognitive processing speed) much more than the consolidated abilities (e.g., those related to cumulative skills and memories) BID4. In other words, the detected changes of linguistic features, including more pauses and decreased short-term memories, could attribute to just normal aging process instead of dementia. Unfortunately, due to the high correlation between dementia and aging, it can be difficult to disentangle symptoms are caused by dementia or aging BID24. Age is therefore a confounding factor in detecting dementia. The effects of confounding factors are hard for traditional machine learning algorithms to isolate, and this is largely due to sampling biases in the data. For example, some algorithms predict higher risk of criminal recidivism for people with darker skin colors BID15, others identify images of smiling Asians as blinking BID19, and GloVe word embeddings can project European-American names significantly closer to the words like'pleasant' than African-American names BID3. It is preferable for classifiers to make decisions without biasing too heavily on demographic factors, and therefore to isolate the effects of confounding factors. However, as we will show in Experiments, traditional neural network classifiers bias on age to infer dementia; this can lead to otherwise avoidable false positives and false negatives that are especially important to avoid in the medical domain. Graphically, if both age A and dementia D cause changes in a feature X, the is a v-structure BID17 A → X ← D which is activated upon observing X. In other words, the confounder A affects P (D|X) if we train the classifier in traditional ways, which is to collect data points {(X, D) (i) } and to learn an inference model P (D|X) approximating the affected P (D|X).Traditionally, there are several ways to eliminate the effects of confounding factors A.Controlling A gives a posterior distribution P (D|X, A)P (A). This is unfortunately unrealistic for small, imbalanced clinical datasets, in which sparsity may require stratification. However, the stratified distributions P (D|X, A) can be far from a meaningful representation of the real world (as we will show, e.g., in FIG3). Moreover, a discrepancy in the sizes of age groups can skew the age prior P (A), which would seriously inhibit the generalizability of a classifier. Controlling X Conducting a randomized control trial (RCT) on X removes all causal paths leading "towards" the variable X, which gives a de-confounded dataset P (D|do(X)) according to the notation in BID27. However, RCTs on X are even less practical because simultaneously controlling multiple features produces exponential number of scenarios, and doing this to more than 400 features require far more data points than any available dataset. Pre-adjusting X according to a pre-trained model X = f (A) per feature could also approximately generate the dataset P (D|do(X)). However, such a model should consider participant differences, otherwise interpolating using a fixed age A would give exactly the same features for everybody. The participant differences, however, are best characterized via X, which are the values you want to predict. To overcome the various problems with these methods, we let our classifiers be aware of cognitive impairments while actively filtering out any information related to aging. This is a fair representation learning framework that protects age as a "sensitive attribute".Fair representation learning frameworks can be used to train classifiers to equally consider the subjects with different sensitive attributes. A sensitive attribute (or "protected attribute") can be race, age, or other variables whose impact should be ignored. In the framework proposed by BID32, classifiers were penalized for the differences in classification probabilities among different demographic groups. After training, the classifiers produced better demographic similarities while compromising only a little overall accuracy. To push the fair representation learning idea further, adversarial training can be incorporated. BID9 introduced generative adversarial networks, in which a generator and a discriminator are iteratively optimized against each other. Incorporating adversarial training, BID22 proposed a framework to learn a latent representation of data in order to limit its adversary's ability to classify based on the sensitive attributes. However, these approaches to fair representation learning only handle binary attributes. E.g., BID22 binarized age. To apply to cognitive impairments detection, we want to represent age on a continuous scale (with some granularity if necessary). We formulate a fairness metric for evaluating the ability of a classifier to isolate a continuous-valued attribute. We also propose four models that compress high-dimensional feature vectors into low-dimensional representations which encrypt age from an adversary. We show empirically that our models achieve better fairness metrics than baseline deep neural network classifiers, while compromising accuracies by as little as 2.56% and 2.25% on our two empirical datasets, respectively. There are many measures of entanglement between classifier outcomes and specific variables. We briefly review some relevant metrics, and then propose ours. Correlation (Pearson, Spearman, etc.) is often used to compare classification outputs with component input features. To the extent that these variables are stochastic, several information theoretic measures could be applied, including Kullback-Leibler divergence and Jensen-Shannon divergence. These can be useful to depict characteristics of two distributions when no further information about available data is given. Mutual information can depict the extent of entanglement of two random variables. If we treat age (A) and dementia (D) as two random variables, then adopting the approach of BID18 gives an estimation of I(A, D). However, given the size of clinical datasets, it can be challenging to give precise estimations. An alternative approach is to assume that these variables fit into some probabilistic models. For example, we might assume the age variable A, dementia indicator variable D, and multi-dimensional linguistic feature X fit into some a priori model (e.g., the v-structure mentioned above, A → X ← D), then the mutual information between A and D is: DISPLAYFORM0 where the entropy of age H A and of cognitive impairment H D remain constant with respect to the input data X, and DISPLAYFORM1 However, this marginalized probability is difficult to approximate well, because the accuracy of the term p(A|X) relies on the ability of our model to infer age from features, and FORMULA20 it is hard to decide on a good prior distribution on linguistic features p(X). We want to make the model agnostic to age, leading to a meaningless mutual information in the'ideal' case. In our frameworks, we do not assume specific graphical models that correlate confounds and outcomes, and we propose more explainable metrics than the traditional statistical ones. The literature in fairness representation learning offers several metrics for evaluating the extent of bias in classifiers. Generally, the fairer the classifier is, the less entangled the are with respect to some protected features. Demographic parity BID32 stated that the fairest scenario is reached when the composition of the classifier outcome for the protected group is equal to that of the whole population. While generally useful, this does not apply to our scenario, in which there really are more elderly people suffering from cognitive impairments than younger people (see FIG3).Cross-entropy loss BID5 used the binary classification loss of an adversary that tried to predict sensitive data from latent representations, as a measure of fairness. This measure can only apply to those models containing an adversary component, not traditional classifiers. Moreover, this loss also depends on the ability of the adversary network. For example, a value of this loss could indicate confusing representations (so sensitive information are protected well), but it could also indicate a weak adversary. Equalized odds BID12 proposed a method in which false positive rates should be equal across groups in the ideal case. BID22 defined fairness distance as the absolute difference in false positive rates between two groups, plus that of the false negative rates: DISPLAYFORM0 where p a and n a correspond to the false positive rate and false negative rate, respectively, with sensitive attribute a = 0 (a = 1). We propose an extension of the metric used by BID22 to continuous sensitive attributes, suitable for evaluating an arbitrary two-class classifier., and classifier C. In age-indep-autoencoder and age-indep-entropy FIG0, a reconstructor R tries to reconstruct input data from the hidden representation. In age-indep-consensus-nets FIG0 ), a discriminator D tells apart from which modality the representation originates. First, groups of age along a scale are divided so that each group has multiple participants with both positive and negative diagnoses, respectively. Let a be the age group each participant is in. Then, we aim for the expected false positive (FP) rates of the classifier to be as constant as possible across age groups. This applies likewise to the false negative (FN) rates. To measure their variability, we use their sum of differences against the mean. DISPLAYFORM0 wherex represents the mean of variable x. Special cases To illustrate the nature of our metric, we apply it to several special cases, i.e.:1. When there is only one age group, our fairness metric has its best possible value: ∆ eo = 0. 2. When there are only two age groups, our metric equals that of BID22. 3. In the extreme case where there are as many age groups as there are sample points (assuming there are no two people with identical ages but with different diagnoses), our metric becomes less informative, because the empirical expected false positive rates of that group is either 0 or 1. This is a limitation of our metric, and is the reason that we limit the number of age groups to accommodate the size of the training dataset. Bounds Our metric is bounded. The lower bound, 0, is reached when all false positive rates are equal and when all false negative rates are equal across age groups. Letting N a be the number of age groups divided, an upper bound for ∆ (Na) eo is N a for any better-than-trivial binary classifier. The detailed proof is included in the Appendix. Disentanglement Our fairness metric illustrates disentanglement. A higher ∆ (N) eo corresponds to a higher variation of incorrect predictions by the classifier across different age groups. Therefore, a lower value of ∆ (N) eo is desired for classifiers isolating the effects of age to a better extent. Throughout this paper, we use the terms'fairness','disentanglement', and'isolation' interchangeably. We explain a few design choices here, namely linearity and indirect optimization. eo to be as linear as possible, for explainability of the fairness score itself. This eliminates possible scores consisting of higher order terms of FP / FN rates. eo, we encourage the hidden representations to be age-agnostic (we will explain how to set up age agnostic models in the following section). On the other hand, FP / FN rates are not differentiable after all. In this section, we describe four different ways of building representation learning models, which we call age-indep-simple, age-indep-autoencoder, age-indep-consensus-net, and age-indep-entropy. The simplest model consists of an interpreter network I to compress high-dimensional input data, x, to low-dimensional representations: z = I(x) An adversary A tries to predict the exact age from the representation: DISPLAYFORM0 A classifier C estimated the probability of label (diagnosis) based on the representation: for minibatch x in training data X do 4: DISPLAYFORM1 DISPLAYFORM2 Calculate L a, L c 6: DISPLAYFORM3 For optimization, we set up two losses: the classification negative log likelihood loss L c and the adversarial (L2) loss L a, where: DISPLAYFORM4 We want to train the adversary to minimize the L2 loss, to train the interpreter to maximize it, and to train the classifier (and interpreter) to minimize classification loss. Overall, DISPLAYFORM5 The training steps are taken iteratively, as in previous work BID9. The age-indep-autoencoder structure is similar to BID22, and can be seen as an extension from the age-indep-simple structure. Similar to age-indep-simple, there is an interpreter I, an adversary A, and a classifier C network. The difference is that there is a reconstructor network R that attempts to recover input data from hidden representations: DISPLAYFORM0 The loss functions are set up as: DISPLAYFORM1 Overall, we want to train both the interpreter and the reconstructor to minimize the reconstruction loss term, in additional to all targets mentioned in the age-indep-simple network. DISPLAYFORM2 The detailed algorithm is similar to Algorithm 1 and is in the Appendix. This is another extension from the age-indep-simple structure, borrowing an idea from consensus networks BID33, i.e., that agreements between multiple modalities can in representations that are beneficial for classification. By examining the performance of age-indepconsensus-net, we would like to see whether agreement between multiple modalities of data can be trained to be disentangled from age. Similar to age-indep-simple structures, there is also an adversary A and a classifier C. The interpreter, however, is replaced with several interpreters I 1..M, each compressing a subset of the input data ("modality") into a low-dimensional representation. The key of age-indep-consensusnetwork models is that these representations are encouraged to be indistinguishable. For simplicity, we randomly divide the input features into three modalities (M = 3) with equal (±1) features. A discriminator D tries to identify the modality from which the representation comes: DISPLAYFORM0 The loss functions are set up as: DISPLAYFORM1 Overall, we want to iteratively optimize the networks: DISPLAYFORM2 L c and max DISPLAYFORM3 The detailed algorithm is in the Appendix. Note that we do not combine the consensus network with the reconstructor because they do not work well with each other empirically. In one of the experiments by BID34, each interpreter I m is paired with a reconstructor R m and the performance decreases dramatically. The reconstructor encourages hidden representations to retain the fidelity of data, while the consensus networks urges hidden representations to keep only the information common among modalities, which prohibits the reconstructor and consensus mechanism to function together. The fourth model we apply to fair representation learning is motivated by categorical GANs , where information theoretic metrics characterizing the confidences of predictions can be optimized. This motivates an additional loss function term; i.e., we want to encourage the interpreter to increase the uncertainty (i.e., to minimize the entropy) while letting the adversary become more confident in predicting ages from representations. Age-indep-entropy models have the same network structures as age-indep-autoencoder, except that instead of predicting the exact age, the adversary network outputs the probability of the sample age being larger than the mean: DISPLAYFORM0 This enables us to define the empirical entropy H[p] = E x plog 1 p, which describes the uncertainty of predicting age. Formally, the loss functions are set up as follows: DISPLAYFORM1 where λ H is a hyper-parameter. For comparison, we also include two variants, namely the ageindep-entropy (binary) and age-indep-entropy (Honly) variants, each keeping only one of the two terms in L a. In our experiments, we show that these two terms in L a are better applied together. Overall, the training procedure is the same as age-indep-autoencoder and algorithm pseudocode is in the Appendix: min DISPLAYFORM2 All models are implemented in PyTorch BID26, optimized with Adam BID16 with initial learning rate of 3 × 10 −4, and L2 weight decay 10. For simplicity, we use fully connected networks with ReLU activations BID25 and batch normalization BID14 before output layers, for all interpreter, adversary, classifier, and discriminator networks. Our frameworks can be applied to other types of networks in the future. DementiaBank DementiaBank 1 is the largest available public dataset for assessing cognitive impairments using speech, containing 473 narrative picture descriptions from subjects aged between 45 and 90 BID2. In each sample, a participant talks about what is happening in a clinically validated picture. There is no time limit in each session, but the average description lasts about a minute. 79 samples are excluded due to missing age information. In the remaining data samples, 182 are labeled'control', and 213 are labeled'dementia'. All participants have mini-mental state estimation (MMSE) scores BID6 ) between 1 and 30 2. Of all data samples containing age information, the mean is 68.26 and standard deviation is 9.00. The Famous People dataset BID1 contains 252 transcripts from 17 people (8 with dementia including Gene Wilder, Ronald Reagan and Glen Campbell, and 9 healthy controls including Michael Bloomberg, Woody Allen, and Tara VanDerveer), collected and transcribed from publicly available speech data (e.g., press conferences, interviews, debatse, talk shows). Seven data samples are discarded due to missing age information. Among the remaining samples, there are 121 labeled as control and 124 as impaired. Note that the data samples were gathered across a wide range of ages (mean 59.25, standard deviation 13.60). For those people diagnosed with dementia, there are data samples gathered both before and after the diagnosis, and all of which are labeled as'dementia'. The Famous People dataset permits for early detection several years before diagnosis, which is a more challenging classification task than DementiaBank. Older participants in both DementiaBank FIG3 ) and the Famous People dataset FIG3 ) are more likely to have cognitive impairments. eo and ∆eo ) of several traditional classifiers. DNN is the baseline used to benchmark our neural network based representation learning models. We extract 413 linguistic features from the narrative descriptions and their transcripts. These features were previously identified as the most useful for this task BID28 BID7 BID21 BID13. Each feature is z-score normalized. Acoustic: mean, variance, skewness, and kurtosis of the first 42 cepstral coefficients. Speech fluency: pause-word ratio, utterance length, number and lengths of filled/unfilled pauses. Lexical: cosine similarity between pairs of utterances, word lengths, lexical richness (movingaverage type-token ratio, Brunet's index, and Honoré's statistics BID10).PoS: Number of occurrences of part-of-speech tags, tagged by SpaCy 3.Syntactic and semantic: occurrences of context-free grammar phrase types, parsed by Stanford CoreNLP BID23, and Yngve depth statistics BID31. As part of expository data analysis, we show that these linguistic features contain information indicating age. Simple fully connected neural networks can predict age with mean absolute error of 15.5 ± 1.3 years (on DementiaBank 4) and 14.3 ± 2.5 years (on the Famous People dataset 5). This indicates that even simple neural networks are able to infer information about age from linguistic features. Neural classifiers can therefore also easily bias on age, given the utility of age in downstream tasks. We first set up benchmarks for our classifiers. We evaluate several traditional classifiers with our fairness metrics (∆ eo, corresponding to dividing ages into N = 2 and N = 5 groups respectively). The 6 are listed in Table 1. A DNN is used as the baseline because all our models are based on neural networks, and DNN classifiers have had the best (or statistically indistinguishable from the best) accuracy on the DementiaBank and Famous People datasets. We evaluate the performances of our four proposed neural networks against the DNN baseline. As an additional ablation study, two variants of age-indep-entropy are also evaluated. TAB1: Evaluation of our representation learning models. The "age-indep" prefix are replaced with "*" in model names. age-indep-simple and age-indep-autoencoder have better disentanglement scores, while the rest two models could have better accuracy. Accuracy The fair representation learning models compromise accuracy, in comparison to DNN baselines. This confirms that part of the classification power of DNNs come from biasing with regards to age. On DementiaBank, the age-indep-autoencoder reduces accuracy the least (only 2.56% in comparison to the DNN baseline). On the Famous People data, age-indep-consensus and age-indep-entropy models compromise accuracies by only 2.25% and 2.75% respectively, which are not statistically different from the DNN baseline 7.Disentanglement In comparison to DNN baselines, our fair representation learning models improve disentanglement/fairness 8, the improvements are mostly significant when measured by the two-group scores ∆eo. Also, the five-group scores ∆eo are less stable for both datasets, and the scores in the Famous People have higher variances than in DementiaBank. Following is an explanation. DementiaBank has ∼400 data samples. In 5-fold cross validation, each of the five age groups has only ∼16 samples during evaluation. Famous People data contains ∼250 samples, which increases the variance. When the number of groups, N of ∆ (N) eo, is kept small (e.g., ∼100 samples per label per group, as in DementiaBank N = 2), the fairness metrics are stable. The model age-indep-entropy is best used with a loss function containing both the binary classification term and the uncertainty minimization term. As shown in TAB1, although having similar fairness metrics 9, the two variants with only one term could have lower accuracy than age-indep-entropy. In general, age-indep-simple and age-indep-autoencoder achieve the best fairness metrics. Noticeably, the better of them surpass traditional classifiers in both ∆ Here, we identify the problem of entangling age in the detection of cognitive impairments. After explaining this problem with causality diagrams, we formulate it into a fair representation learning task, and propose a fairness score to measure the extent of disentanglement. We put forward four fair representation learning models that learn low-dimensional representations of data samples containing as little age information as possible. Our best model improves upon the DNN baseline in our fairness metrics, while compromising as little accuracy as 2.56% (on DementiaBank) and 2.25% (on the Famous People dataset).7 p = 0.20, 0.16 on 38-DoF one-tailed t-tests, respectively. 8 On DementiaBank, p = 0.01 and 0.03 for age-indep-simple and age-indep-entropy on ∆ eo respectively; these are significant. p = 0.08 and 0.09 on age-indep-autoencoder and age-indep-consensus-net on ∆ eo respectively; these are marginally significant. However, these differences are not as significant on ∆ Proof of Theorem For each of the age groups: |p a −p| + |n a −n| ≤ max{|p a − 0| + |n a − 0|, |p a − 0.5| + |n a − 0.5|} ≤ max{0.5, 1} = 1 Summing up the N a age groups in our upper bound N a for non-trivial classifiers. Following are the pseudo-code algorithms for our remaining three models; age-indep-AutoEncoder, age-indep-ConsensusNetworks, and age-indep-Entropy. for minibatch x in training data X do for minibatch x in training data X do

Show that age confounds cognitive impairment detection + solve with fair representation learning + propose metrics and models.

represent the short summary for finding the corresponding scientific article (long)
For example, 'Proposed architecture to solve morphological agreement task' should be close to 'Neural conversational models are widely used in applications like personal assistants and chat bots. These models seem to give better performance when operating on word level. However, for fusion languages like French, Russian and Polish vocabulary size sometimes become infeasible since most of the words have lots of word forms. We propose a neural network architecture for transforming normalized text into a grammatically correct one. Our model efficiently employs correspondence between normalized and target words and significantly outperforms character-level models while being 2x faster in training and 20\% faster at evaluation. We also propose a new pipeline for building conversational models: first generate a normalized answer and then transform it into a grammatically correct one using our network. The proposed pipeline gives better performance than character-level conversational models according to assessor testing. Neural conversational models BID18 are used in a large number of applications: from technical support and chat bots to personal assistants. While being a powerful framework, they often suffer from high computational costs. The main computational and memory bottleneck occurs at the vocabulary part of the model. Vocabulary is used to map a sequence of input tokens to embedding vectors: one embedding vector is stored for each word in vocabulary. English is de-facto a standard language for training conversational models, mostly for a large number of speakers and simple grammar. In english, words usually have only a few word forms. For example, verbs may occur in present and past tenses, nouns can have singular and plural forms. For many other languages, however, some words may have tens of word forms. This is the case for Polish, Russian, French and many other languages. For these languages storing all forms of frequent words in a vocabulary significantly increase computational costs. To reduce vocabulary size, we propose to normalize input and output sentences by putting them into a standard form. Generated texts can then be converted into grammatically correct ones by solving morphological agreement task. This can be efficiently done by a model proposed in this work. Our contribution is two-fold:• We propose a neural network architecture for performing morphological agreement in fusion languages such as French, Polish and Russian (Section 2).• We introduce a new approach to building conversational models: generating normalized text and then performing morphological agreement with proposed model (Section 3); In this section we propose a neural network architecture for solving morphological agreement problem. We start by formally defining the morphological agreement task. Consider a grammatically correct sentence with words [a 1, a 2, . . ., a K]. Let S(a) be a function that maps any word to its standard form. For example, S("went") = "go". Goal of morphological agreement task is to learn a mapping from normalized sentence [S(a 1), S(a 2),..., S(a K)] = [a n 1,,a n 2, . . ., a n K] to initial sentence [a 1, a 2, . . ., a K]. Interestingly, reverse mapping may be performed for each word independently using specialized dictionaries. Original task, however, needs to consider dependencies between words in sequence in order to output a coherent text. An important property of this task is that the number of words, their order and meaning are explicitly contained in input sequence. To employ this knowledge, we propose a specific neural network architecture illustrated in FIG0. Network operates as follows: first all normalized words are embedded using the same character-level LSTM encoder. The goal of the next step is to incorporate global information about other words to embedding of each word. To do so we pass word embedding sequence through a bidirectional LSTM BID7. This allows new embeddings to get information from all other words: information about previous words is brought by forward LSTM and information about subsequent words is taken from backward LSTM. Finally, new embeddings are decoded with character-level LSTM decoders. At this stage we also added attention BID1 over input characters of corresponding words for better performance. High-level overview of this model resembles sequence-to-sequence BID17 network: model learns to map input characters to output characters for each word using encoder-decoder sceme. The difference is that bidirectional neural network is used to distribute information between different words. Unlike simple character-level sequence-to-sequence model, our architecture allows for much faster evaluation, since encoding and decoding phases can be done in parallel for each word. Also, one of the main advantages of our approach is that information paths from inputs to outputs are much shorter which leads to slower performance degradation as input length increases (see Section 5.3). As discussed above, we propose a two stage approach to build a neural conversational model: first generate normalized answer using normalized question and then apply Concorde model to obtain grammatically correct response. In this section we discuss a modification of Concorde model for conditioning its output on question's morphological features. A modification of Concorde model that we call Q-Concorde uses two sources of input: question and normalized answer. Question is first embedded into a single vector with character-level RNN. This vector may carry important morphological information such as time, case and plurality of questions. Question embedding is then mixed with answer embeddings using linear mapping. The final model is shown in FIG2. DISPLAYFORM0 Most frequently used models for sequence to sequence mapping rely on generating embedding vector that contains all information about input sequence. Reconstruction solely from this vector usually in worse performance as length of the output sequence increases. Attention BID19, BID14, BID1 ) partially fixes this problem, though information bottleneck of embedding vector is still high. Encoder-decoder models have mostly been applied to tasks like speech recognition BID6, BID7, BID5 ), machine translation BID1, BID17, BID2 ) and neural conversational models BID18, BID16 ).Some works have tried to perform decomposition of input sequence to obtain shorter information paths. In input is first processed character-wise and then embeddings that correspond to word endings are used for sequence-to-sequence model. This modification makes input-output information paths shorter which leads to better performance. Word inflection is a most similar task to ours, but in this task model is asked to generate a specific word form while we want our model to automatically select desired word forms. BID3 proposed a supervised approach to predicting the set of all word forms by generating transformation rules from known inflection tables. They also propose to use Conditional Random Fields for unseen base forms. Some authors have also tried to apply neural networks for this problem. BID0 and BID4 propose to use bidirectional LSTM to encode the word. Then BID4 uses different decoders for different word forms, while BID0 suggests to have one decoder and to attach morphological features to its input. Besides recurrent networks, there has been an attempt to use convolutional networks. BID15 based his work on BID4 and proposed to first pass raw data through convolutional layers and then to pass them through recurrent encoder. BID9 BID11 library. We leave first 20 words from each sentence to reduce computational costs. Concorde and Q-Concorde models consist of 2-layer LSTM encoder and decoder with hidden size 512. We compare our model to three baselines: unigram charRNN, bigram charRNN and hierarchical model. Unigram and bigram models are standard sequence-to-sequence models with attention BID14 that operate with characters or pairs of characters as tokens. We use 2-layer LSTM as an encoder. Decoder consists of 2-layer LSTM followed by attention layer and another recurrent layer. The third baseline is a hierarchical model motivated by: we first embed each word using recurrent encoder and then compute sentence embedding by running word-level encoder on these embeddings. For baselines we use layer size of 768 which in a comparable number of parameters for all models. We train models with Adam BID10 optimizer in batch size 16 with learning rate 0.0002 that halves after each 50k updates. We terminate training after 300k updates which is enough for all models to converge. To evaluate our model we used French, Russian an Polish corpuses from OpenSubtitles 2 database. We performed morphological agreement for each subtitle line independently. We estimated potential vocabulary size reduction from normalization by selecting words that appeared more than 10 times in first 10M examples. This lead to 2.5 times reduction for Polish language, 2.4 for Russian, and 1.8 for French. We evaluated our model in two metrics: word and sentence accuracies. Word accuracy shows a fraction of words that were correctly generated. Sentence accuracy corresponds to the fraction of sentences that were transformed without any mistake. Results are reported in TAB1. From four models that we compared, our model gave the best performance among all datasets, while second best model was hierarchical model. We inspected our model to show some examples where it was able to infer plural form and gender for unseen words TAB2 ). For Russian language we found out that the model was able to learn some rare rules like changing the letter я to й when going to plural form in some words: один заяц, два зайца (one rabbit, two rabbits).We can also see that our model can infer gender from words. To show that we chose feminine, masculine and neuter words and asked the model to perform agreement with word "one". This word changes its spelling for different genders in French, Polish, and Russian. Results presented in TAB3 suggest that model can indeed correctly solve this task by setting correct gender to the numeral. TAB4 we show on full sentences. Interestingly, on quite a complex Russian example our model was able to perform agreement. To select the correct form of word соседнем (neighbouring), network had to use multiple markers from different parts of a sentence: gender from подъезд (entrance) and case from в (in). As a motivation for our model we argued that making shorter input-output paths may reduce information load of the embedding vector. To check this hypothesis we computed average sentence accuracy for different input lengths and reported in FIG2.We can clearly see that all baseline models perform worse as the input length increases. However, this is not the case for our model -while character-level models perform with almost 0% accuracy when input is 100 characters long, our model still gives similar performance as for short sentences. This can be explained by the way in which models use embedding vectors. Baseline models have to share embedding capacity between all words in a sentence. Our model, however, has a separate embedding for each word and does not require the whole sentence to be squeezed into one vector. It is also clear that character-level models perform better for short sequences (about 33% of the test set). This may be the case since the capacity of the embedding vector is not fully used for them. Despite being worse for short inputs, our model can still handles many important cases quite well including those discussed in Section 5.2. To evaluate our conversation model we constructed a corpus of question-answer pairs from web site Otvet.mail.ru -general topic Russian service for questions and answers (analogue of Quora.com). The uniqueness of this corpus is that it contains general knowledge questions that allow the trained model to answer questions about movies, capitals, etc. This requires many rare entity-related words to be in the vocabulary which makes it extremely large without normalization. First we compared Q-Concorde and Concorde models to show that Q-Concorde can indeed grasp important morphological features form a context. We also trained baseline models with context concatenated to input sentence (with a special delimiter in between). word and sentence accuracies are reported in TAB5. Again, Concorde model was able to outperform baselines even though it didn't have access to the context. Also, Q-Concorde model was able to improve Concorde's performance. We inspected cases on which Q-Concorde model showed better performance than Concorde TAB6 ). In example 1 of this table, question was asked about a single object. Q-Concorde model used singular form, while Concorde used plural. Q-Concorde was also able to successfully carry correct case (example 2) and time (example 3) from the question. Some mistakes made by Q-Concorde model are shown in TAB7. In example 1, Q-Concorde wasn't able to decide whether to use polite form or not and used one word in a less polite form than another. An important property of Q-Concorde model is that it can generate different texts depending on lexical features of a question. For example, in TAB8 we changed question's tense from present simple to past simple ("what do you do?" and "what did you do?"). The model correctly generated answer in corresponding tense. We also tried to change gender of a word "did" in a question: from masculine делал ) to feminine далала. Our model used the correct gender to generate the answer. While model generated grammatically correct answers in all three cases, in the third case (past simple, masculine) model answered in less common form with a meaning that differs from expected one. Finally, we apply Q-Concorde model to a proposed pipeline for training conversational models. We compare our model with a 3-layer character-level sequence-to-sequence model which was trained on grammatically correct sentences. For generating diverse answers we train two models: one to predict answer given question and the other to predict question given answer, as suggested in BID16. This allows us to discard answers that are too general. To compare two models we set an experiment environment where assessors were asked to select one of two possible answers to the given question: one was generated by character-wise model and another was generated by our pipeline. Assessors did not know the order in which cases were shown and so they did not know which model generated the text. In 62.1% cases assessors selected the proposed model, and in the remaining 37.9% assessors preferred character-wise model. We noticed that time for processing one batch is much higher for character-level models since they need to process longer sequences sequentially. In TAB9 we report time for forward and backward pass of one batch (16 objects) and other important computational characteristics. We measured this time on GeForce GTX TITAN X graphic card. It turns out that proposed models have comparable evaluation time, but train faster than unigram and hierarchical models. In this paper we proposed a neural network model that can efficiently employ relationship between input and output words in morphological agreement task. We also proposed a modification for this model that uses context sentence. We apply this model for neural conversational model in a new pipeline: we use normalized question to generate normalized answer and then apply proposed model to obtain grammatically correct response. This model showed better performance than character level neural conversational model based on assessors responses. We achieved significant improvement comparing to character-level, bigram and hierarchical sequenceto-sequence models on morphological agreement task for Russian, French and Polish languages. Trained models seem to understand main grammatical rules and notions such as tenses, cases and pluralities.'


Question: Represent the article for finding a single-sentence TLDR

Question: Much of the focus in the design of deep neural networks had been on improving accuracy, leading to more powerful yet highly complex network architectures that are difficult to deploy in practical scenarios. As a , there has been a recent interest in the design of quantitative metrics for evaluating deep neural networks that accounts for more than just model accuracy as the sole indicator of network performance. In this study, we continue the conversation towards universal metrics for evaluating the performance of deep neural networks for practical on-device edge usage by introducing NetScore, a new metric designed specifically to provide a quantitative assessment of the balance between accuracy, computational complexity, and network architecture complexity of a deep neural network. In what is one of the largest comparative analysis between deep neural networks in literature, the NetScore metric, the top-1 accuracy metric, and the popular information density metric were compared across a diverse set of 60 different deep convolutional neural networks for image classification on the ImageNet Large Scale Visual Recognition Challenge (ILSVRC 2012) dataset. The evaluation across these three metrics for this diverse set of networks are presented in this study to act as a reference guide for practitioners in the field. There has been a recent urge in both research and industrial interests in deep learning, with deep performance across a wide variety of applications BID22 22, 11]. However, the practical industrial BID24 deployment bottlenecks associated with the powerful yet highly complex deep neural networks in 22 research literature has become even increasingly visible, and as a , the design of deep neural 23 networks that strike a strong balance between accuracy and complexity become a very hot area of 24 research focus [18, 14, 34, 33, 26, BID31 36]. One of the key challenges in designing practical deep 25 neural networks lies in the difficulties with assessing how well a particular network architecture 26 is striking that balance. One of the most widely cited metrics is the information density metric 27 proposed by BID0, which attempts to measure the relative amount of accuracy given network size. However, information density does not account for computational requirements for performing 29 network inference (e.g., MobileNet has more parameters than SqueezeNet but has lower 30 computational requirements for network inference). Therefore, the exploration and investigation 31 towards universal performance metrics that account for accuracy, architectural complexity, and 32 computational complexity is highly desired as it has the potential to improve network model search 33 and design. In this study, we introduce NetScore, a new metric designed specifically to provide a 34 quantitative assessment of the balance between accuracy, computational complexity, and network low accuracy remain unusable in practical scenarios, regardless how small or fast the network is. Furthermore, we set β = 0.5 and γ = 0.5 since, while architectural and computational complexity are the presented in this study can act as a reference guide for practitioners in the field. The set of deep convolutional neural networks being evaluated in this study are: AlexNet BID22, shown in Fig. 1(right). Similar to the trend observed in Fig. 1

We introduce NetScore, new metric designed to provide a quantitative assessment of the balance between accuracy, computational complexity, and network architecture complexity of a deep neural network.

Represent the short summary for finding the corresponding scientific article (long)

An efficient lifelong learning algorithm that provides a better trade-off between accuracy and time/ memory complexity compared to other algorithms.


Question: represent the article for finding a single-sentence tldr

Question: Deep Neural Networks (DNNs) are vulnerable to adversarial attacks, especially white-box targeted attacks. This paper studies the problem of how aggressive white-box targeted attacks can be to go beyond widely used Top-1 attacks. We propose to learn ordered Top-k attacks (k>=1), which enforce the Top-k predicted labels of an adversarial example to be the k (randomly) selected and ordered labels (the ground-truth label is exclusive). Two methods are presented. First, we extend the vanilla Carlini-Wagner (C&W) method and use it as a strong baseline. Second, we present an adversarial distillation framework consisting of two components: (i) Computing an adversarial probability distribution for any given ordered Top-$k$ targeted labels. (ii) Learning adversarial examples by minimizing the Kullback-Leibler (KL) divergence between the adversarial distribution and the predicted distribution, together with the perturbation energy penalty. In computing adversarial distributions, we explore how to leverage label semantic similarities, leading to knowledge-oriented attacks. In experiments, we test Top-k (k=1,2,5,10) attacks in the ImageNet-1000 val dataset using two popular DNNs trained with the clean ImageNet-1000 train dataset, ResNet-50 and DenseNet-121. Overall, the adversarial distillation approach obtains the best , especially by large margin when computation budget is limited.. It reduces the perturbation energy consistently with the same attack success rate on all the four k's, and improve the attack success rate by large margin against the modified C&W method for k=10. Despite the recent dramatic progress, deep neural networks (DNNs) (; ; ;) trained for visual recognition tasks (e.g., image classification) can be easily fooled by so-called adversarial attacks which utilize visually imperceptible, carefully-crafted perturbations to cause networks to misclassify inputs in arbitrarily chosen ways in the close set of labels used in training (; ; ;), even with one-pixel attacks . The existence of adversarial attacks hinders the deployment of DNNs-based visual recognition systems in a wide range of applications such as autonomous driving and smart medical diagnosis in the long-run. In this paper, we are interested in learning visually-imperceptible targeted attacks under the whitebox setting in image classification tasks. In the literature, most methods address targeted attacks in the Top-1 manner, in which an adversarial attack is said to be successful if a randomly selected label (not the ground-truth label) is predicted as the Top-1 label with the added perturbation satisfying to be visually-imperceptible. One question arises, • The "robustness" of an attack method itself: How far is the attack method able to push the underlying ground-truth label in the prediction of the learned adversarial examples? Table 1 shows the evaluation of the "robustness" of different attack methods. The widely used C&W method does not push the GT labels very far, especially when smaller perturbation energy is aimed using larger search range (e.g., the average rank of the GT label is 2.6 for C&W 9×1000). Consider Top-5, if the ground-truth labels of adversarial examples still largely appear in the Top-5 of the prediction, we may be over-confident about the 100% ASR, . Please see Sec. 4 for detail of experimental settings. Method ASR Proportion of GT Labels in Top-k (smaller is better) Average Rank of GT Labels (larger is better) Top-3 Top-5 Top-10 Top-50 Top-100 C&W9×30 99.9 36.9 50.5 66.3 90.0 95.1 20.4 C&W9×1000 100 71.9 87.0 96.1 99.9 100 2.6 FGSM 80.7 25.5 37.8 52.8 81.2 89.2 44.2 PGD10 100 3.3 6.7 12 34.7 43.9 306.5 MIFGSM10 99.9 0.7 1.9 6.0 22.5 32.3 404.4 especially when some downstream modules may rely on Top-5 predictions in their decision making. But, the three untargeted attack approaches are much better in terms of pushing the GT labels since they are usually move against the GT label explicitly in the optimization, but their perturbation energies are usually much larger. As we shall show, more "robust" attack methods can be developed by harnessing the advantages of the two types of attack methods. In addition, the targeted Top-1 attack setting could limit the flexibility of attacks, and may lead to less rich perturbations. To facilitate explicit control of targeted attacks and enable more "robust" attack methods, one natural solution, which is the focus of this paper, is to develop ordered Top-k targeted attacks which enforce the Top-k predicted labels of an adversarial example to be the k (randomly) selected and ordered labels (k ≥ 1, the GT label is exclusive). In this paper, we present two methods of learning ordered Top-k attacks. The basic idea is to design proper adversarial objective functions that in imperceptible perturbations for any test image through iterative gradient-based back-propagation. First, we extend the vanilla Carlini-Wagner (C&W) method and use it as a strong baseline. Second, we present an adversarial distillation (AD) framework consisting of two components: (i) Computing an adversarial probability distribution for any given ordered Top-k targeted labels. (ii) Learning adversarial examples by minimizing the Kullback-Leibler (KL) divergence between the adversarial distribution and the predicted distribution, together with the perturbation energy penalty. The proposed AD framework can be viewed as applying the network distillation frameworks (; ;) for "the bad" induced by target adversarial distributions. To compute a proper adversarial distribution for any given ordered Top-k targeted labels, the AD framework is motivated by two aspects: (i) The difference between the objective functions used by the C&W method and the three untargeted attack methods (Table 1) respectively. The former maximizes the margin of the logits between the target and the runner-up (either GT or ResNet-50. AD is better than the modified C&W method (CW *). The thickness represents the 2 energy (thinner is better). Please see Sec. 4 for detail of experimental settings. not), while the latter maximizes the cross-entropy between the prediction probabilities (softmax of logits) and the one-hot distribution of the ground-truth. (ii) The label smoothing methods ), which are often used to improve the performance of DNNs by addressing the over-confidence issue in the one-hot vector encoding of labels. More specifically, we explore how to leverage label semantic similarities in computing "smoothed" adversarial distributions, leading to knowledge-oriented attacks. We measure label semantic similarities using the cosine distance between some off-the-shelf word2vec embedding of labels such as the pretrained Glove embedding . Along this direction, another question of interest is further investigated: Are all Top-k targets equally challenging for an attack approach? In experiments, we test Top-k (k = 1, 2, 5, 10) in the ImageNet-1000 val dataset using two popular DNNs trained with clean ImageNet-1000 train dataset, ResNet-50 and DenseNet-121 respectively. Overall, the adversarial distillation approach obtains the best . It reduces the perturbation energy consistently with the same attack success rate on all the four k's, and improve the attack success rate by large margin against the modified C&W method for k = 10 (see Fig. 1). We observe that Top-k targets that are distant from the GT label in terms of either label semantic distance or prediction scores of clean images are actually more difficulty to attack. In summary, not only can ordered Top-k attacks improve the "robustness" of attacks, but also they provide insights on how aggressive adversarial attacks can be (under affordable optimization budgets). Our Contributions. This paper makes three main contributions to the field of learning adversarial attacks: (i) The problem in study is novel. Learning ordered Top-k adversarial attacks is an important problem that reflects the robustness of attacks themselves, but has not been addressed in the literature. (ii) The proposed adversarial distillation framework is effective, especially when k is large (such as k = 5, 10). (iii) The proposed knowledge-oriented adversarial distillation is novel. It worth exploring the existing distillation framework for a novel problem (ordered Top-k adversarial attacks) with some novel modifications (knowledge-oriented target distributions as "teachers"). The growing ubiquity of DNNs in advanced machine learning and AI systems dramatically increases their capabilities, but also increases the potential for new vulnerabilities to attacks. This situation has become critical as many powerful approaches have been developed where imperceptible perturbations to DNN inputs could deceive a well-trained DNN, significantly altering its prediction. Assuming full access to DNNs pretrained with clean images, white-box targeted attacks are powerful ways of investigating the brittleness of DNNs and their sensitivity to non-robust yet well-generalizing features in the data. Distillation. The central idea of our proposed AD method is built on distillation. Network distillation is a powerful training scheme proposed to train a new, usually lightweight model (a.k.a., the student) to mimic another already trained model (a.k.a. the teacher). It takes a functional viewpoint of the knowledge learned by the teacher as the conditional distribution it produces over outputs given an input. It teaches the student to keep up or emulate by adding some regularization terms to the loss in order to encourage the two models to be similar directly based on the distilled knowledge, replacing the training labels. Label smoothing can be treated as a simple hand-crafted knowledge to help improve model performance. Distillation has been exploited to develop defense models to improve model robustness. Our proposed adversarial distillation method utilizes the distillation idea in an opposite direction, leveraging label semantic driven knowledge for learning ordered Top-k attacks and improving attack robustness. Adversarial Attack. For image classification tasks using DNNs, the discovery of the existence of visually-imperceptible adversarial attacks was a big shock in developing DNNs. White-box attacks provide a powerful way of evaluating model brittleness. In a plain and loose explanation, DNNs are universal function approximator and capable of even fitting random labels in large scale classification tasks as ImageNet-1000 . Thus, adversarial attacks are generally learnable provided proper objective functions are given, especially when DNNs are trained with fully differentible back-propagation. Many white-box attack methods focus on norm-ball constrained objective functions (; ; ;). The C&W method investigates 7 different loss functions. The best performing loss function found by the C&W method has been applied in many attack methods and achieved strong (; ; . By introducing momentum in the MIFGSM method and the p gradient projection in the PGD method , they usually achieve better performance in generating adversarial examples. In the meanwhile, some other attack methods such as the StrAttack also investigate different loss functions for better interpretability of attacks. Our proposed method leverages label semantic knowledge in the loss function design for the first time. In this section, we first briefly introduce the white-box attack setting and the widely used C&W method under the Top-1 protocol, to be self-contained. Then we define the ordered Top-k attack formulation. To learn ordered Top-k attacks, we present detail of a modified C&W method as a strong baseline and the proposed AD framework. We focus on classification tasks using DNNs. Denote by (x, y) a pair of a clean input x ∈ X and its ground-truth label y ∈ Y. For example, in the ImageNet-1000 classification task, x represents a RGB image defined in the lattice of 224×224 and we have X R 3×224×224. y is the category label and we have Y {1, · · ·, 1000}. Let f (·; Θ) be a DNN pretrained on clean training data where Θ collects all estimated parameters and is fixed in learning adversarial examples. For notation simplicity, we denote by f (·) a pretrained DNN. The prediction for an input x from f (·) is usually defined using softmax function by, where P ∈ R |Y| represents the estimated confidence/probability vector (P c ≥ 0 and c P c = 1) and z(x) is the logit vector. The predicted label is then inferred byŷ = arg max c∈[1,|Y|] P c. The traditional Top-1 protocol of learning targeted attacks. For an input (x, y), given a target label t = y, we seek to compute some visually-imperceptible perturbation δ(x, t, f) using the pretrained and fixed DNN f (·) under the white-box setting. White-box attacks assume the complete knowledge of the pretrained DNN f, including its parameter values, architecture, training method, etc. The perturbed example is defined by, which is called an adversarial example of x if t =ŷ = arg max c f (x) c and the perturbation δ(x, t, f) is sufficiently small according to some energy metric. The C&W Method . Learning δ(x, t, f) under the Top-1 protocol is posed as a constrained optimization problem , n, where E(·) is defined by a p norm (e.g., the 2 norm) and n the size of the input domain (e.g., the number of pixels). To overcome the difficulty (non-linear and non-convex constraints) of directly solving Eqn. 3, the C&W method expresses it in a different form by designing some loss functions L(x) = L(x + δ) such that the first constraint t = arg max c f (x) c is satisfied if and only if L(x) ≤ 0. The best loss function proposed by the C&W method is defined by the hinge loss, which induces penalties when the logit of the target label is not the maximum among all labels. Then, the learning problem is formulated by, subject to x + δ ∈ n, which can be solved via back-propagation with the constraint satisfied via introducing a tanh layer. For the trade-off parameter λ, a binary search will be performed during the learning (e.g., 9 × 1000). It is straightforward to extend Eqn. 3 for learning ordered Top-k attacks (k ≥ 1). Denote by (t 1, · · ·, t k) the ordered Top-k targets (t i = y). We have, minimize subject to t i = arg max Directly solving Eqn. 6 is a difficulty task and proper loss functions are entailed, similar in spirit to the approximation approaches used in the Top-1 protocol, to ensure the first constraint can be satisfied once the optimization is converged (note that the optimization may fail, i.e., attacks fail). 3.3 LEARNING ORDERED TOP-k ATTACKS 3.3.1 A MODIFIED C&W METHOD We can modify the loss function (Eqn. 4) of the C&W method accordingly to solve Eqn. 6. We have, which covers the vanilla C&W loss (Eqn. 4), i.e., when k = 1, CW (x). The C&W loss function does not care where the underlying GT label will be as long as it is not in the Top-k. On the one hand, it is powerful in terms of attack success rate. On the other hand, the GT label may be very close to the Top-k, leading to over-confident attacks (see Tabel. 1). In addition, it is generic for any given Top-k targets. As we will show, they are less effective if we select the Top-k targets from the sub-set of labels which are least like the ground-truth label in terms of label semantics. To overcome the shortcomings of the C&W loss function and In our adversarial distillation framework, we adopt the view of point proposed in the network distillation method that the full confidence/probability distribution summarizes the knowledge of a trained DNN. We hypothesize that we can leverage the network distillation framework to learn the ordered Top-k attacks by designing a proper adversarial probability distribution across the entire set of labels that satisfies the specification of the given ordered Top-k targets, and facilitates explicit control of placing the GT label, as well as top-down integration of label semantics. Consider a given set of Top-k targets, {t 1, · · ·, t k}, denoted by P AD the adversarial probability distribution in which P AD ti > P AD tj (∀i < j) and P AD ti The space of candidate distributions are huge. We present a simple knowledge-oriented approach to define the adversarial distribution. We first specify the logit distribution and then compute the probability distribution using softmax. Denote by Z the maximum logit (e.g., Z = 10 in our experiments). We define the adversarial logits for the ordered Top-k targets by, z where γ is an empirically chosen decreasing factor (e.g., γ = 0.3 in our experiments). For the remaining categories l / ∈ {t 1, · · ·, t k}, we define the adversarial logit by, where 0 ≤ α < z AD t k is the maximum logit that can be assigned to any j, s(a, b) is the semantic similarity between the label a and label b, and is a small position for numerical consideration (e.g., = 1e-5). We compute s(a, b) using the cosine distance between the Glove embedding vectors of category names and −1 ≤ s(a, b) ≤ 1. Here, when α = 0, we discard the semantic knowledge and treat all the remaining categories equally. Note that our design of P AD is similar in spirit to the label smoothing technique and its variants ) except that we target attack labels and exploit label semantic knowledge. The design choice is still preliminary, although we observe its effectiveness in experiments. We hope this can encourage more sophisticated work to be explored. With the adversarial probability distribution P AD defined above as the target, we use the KL divergence as the loss function in our adversarial distillation framework as done in network distillation and we have, L (k) and then we follow the same optimization scheme as done in the C&W method (Eqn. 5). In this section, we evaluate ordered Top-k attacks with k = 1, 2, 5, 10 in the ImageNet-1000 benchmark using two pretrained DNNs, ResNet-50 and DenseNet-121 from the PyTorch model zoo 1. We implement our method using the AdverTorch toolkit 2. Our source code will be released. Data. In ImageNet-1000 , there are 50, 000 images for validation. To study attacks, we utilize the subset of images for which the predictions of both the ResNet-50 and DenseNet-121 are correct. To reduce the computational demand, we randomly sample a smaller subset, as commonly done in the literature. We first randomly select 500 categories to enlarge the coverage of categories, and then randomly chose 2 images per selected categories, ing in 1000 test images in total. Settings. We follow the protocol used in the C&W method. We only test 2 norm as the energy penalty for perturbations in learning (Eqn. 5). But, we evaluate learned adversarial examples in terms of three norms (1, 2 and ∞). We test two search schema for the trade-off parameter λ in optimization: both use 9 steps of binary search, and 30 and 1000 iterations of optimization are performed for each trial of λ. In practice, computation budget is an important factor and less computationally expensive ones are usually preferred. Only α = 1 is used in Eqn. 9 in experiments for simplicity due to computational demand. We compare the under three scenarios proposed in the C&W method : The Best Case settings test the attack against all incorrect classes, and report the target class(es) that was least difficult to attack. The Worst Case settings test the attack against all incorrect classes, and report the target class(es) that was most difficult to attack. The Average Case settings select the target class(es) uniformly at random among the labels that are not the GT. We first test ordered Top-k attacks using ResNet-50 for the four selected k's. Table. 2 summarizes the quantitative and comparisons. For Top-10 attacks, the proposed AD method obtains significantly better in terms of both ASR and the 2 energy of the added perturbation. For example, the proposed AD method has relative 362.3% ASR improvement over the strong C&W baseline for the worst case setting. For Top-5 attacks, the AD method obtains significantly better when the search budget is relatively low (i.e., 9 × 30). For Top-k (k = 1, 2) attacks, both the C&W method and the AD method can achieve 100% ASR, but the AD method has consistently lower energies of the added perturbation, i.e., finding more effective attacks and richer perturbations. Fig. 2 shows some learned adversarial examples of ordered Top-10 and Top-5 attacks. Table 2: Results and comparisons under the ordered Top-k targeted attack protocol using randomly selected and ordered 10 targets (GT exclusive) in ImageNet using ResNet-50. For Top-1 attacks, we also compare with three state-of-the-art untargeted attack methods, FGSM , PGD and MIFGSM . 10 iterations are used for both PGD and MIFGSM. Intuitively, we understand that they should not be equally difficult. We conduct some experiments to test this hypothesis. In particular, we test whether the label semantic knowledge can help identify the weak spots of different attack methods, and whether the proposed AD method can gain more in those weak spots. We test Top-5 using ResNet-50 3. Table. 3 summarizes the . We observe that for the 9 × 30 search budget, attacks are more challenging if the Top-5 targets are selected from the least-like set in terms of the label semantic similarity (see Eqn. 9), or from the lowest-score set in terms of prediction scores on clean images. To investigate if the observations from ResNets hold for other DNNs, we also test DenseNet-121 in ImageNet-1000. We test two settings: k = 1, 5 4. Overall, we obtain similar . . The proposed AD method has smaller perturbation energies and "cleaner" (lower-entropy) prediction distributions. Note that for Top-10 attacks, the 9 × 30 search scheme does not work (see Table. 2). Table 3: Results of ordered Top-5 targeted attacks with targets being selected based on (Top) label similarity, which uses 5 most-like labels and 5 least-like labels as targets respectively, and (Bottom) prediction score of clean image, which uses 5 highest-score labels and 5-lowest score labels. In both cases, GT labels are exclusive. Top-1 targeted attack protocol using randomly selected and ordered 5 targets (GT exclusive). For Top-1 attacks, we also compare with three state-of-the-art untargeted attack methods, FGSM , PGD and MIFGSM . 10 iterations are used for both PGD and MIFGSM. This paper proposes to extend the traditional Top-1 targeted attack setting to the ordered Top-k setting (k ≥ 1) under the white-box attack protocol. The ordered Top-k targeted attacks can improve the robustness of attacks themselves. To our knowledge, it is the first work studying this ordered Top-k attacks. To learn the ordered Top-k attacks, we present a conceptually simple yet effective adversarial distillation framework motivated by network distillation. We also develop a modified C&W method as the strong baseline for the ordered Top-k targeted attacks. In experiments, the proposed method is tested in ImageNet-1000 using two popular DNNs, ResNet-50 and DenseNet-121, with consistently better obtained. We investigate the effectiveness of label semantic knowledge in designing the adversarial distribution for distilling the ordered Top-k targeted attacks. Discussions. We have shown that the proposed AD method is generally applicable to learn ordered Top-k attacks. But, we note that the two components of the AD framework are in their simplest forms in this paper, and need to be more thoroughly studied: designing more informative adversarial distributions to guide the optimization to learn adversarial examples better and faster, and investigating loss functions other than KL divergence such as the Jensen-Shannon (JS) divergence or the Earth-Mover distance. On the other hand, we observed that the proposed AD method is more effective when computation budget is limited (e.g., using the 9 × 30 search scheme). This leads to the theoretically and computationally interesting question whether different attack methods all will work comparably well if the computation budget is not limited. Of course, in practice, we prefer more powerful ones when only limited computation budget is allowed. Furthermore, we observed that both the modified C&W method and the AD method largely do not work in learning Top-k (k ≥ 20) attacks with the two search schema (9 × 30 and 9 × 1000). We are working on addressing the aforementioned issues to test the Top-k (k ≥ 20) cases, thus providing a thorough empirical answer to the question: how aggressive can adversarial attacks be?

Represent the TLDR for finding the corresponding scientific article

We revisit the simple idea of pruning connections of DNNs through $\ell_1$ regularization achieving state-of-the-art results on multiple datasets with theoretic guarantees.