Question:
represent the text to find a very short summary (1-2 sentences max)
------
E.g. There are many differences between convolutional networks and the ventral visual streams of primates. For example, standard convolutional networks lack recurrent and lateral connections, cell dynamics, etc. However, their feedforward architectures are somewhat similar to the ventral stream, and warrant a more detailed comparison. A recent study found that the feedforward architecture of the visual cortex could be closely approximated as a convolutional network, but the ing architecture differed from widely used deep networks in several ways. The same study also found, somewhat surprisingly, that training the ventral stream of this network for object recognition ed in poor performance. This paper examines the performance of this network in more detail. In particular, I made a number of changes to the ventral-stream-based architecture, to make it more like a DenseNet, and tested performance at each step. I chose DenseNet because it has a high BrainScore, and because it has some cortex-like architectural features such as large in-degrees and long skip connections. Most of the changes (which made the cortex-like network more like DenseNet) improved performance. Further work is needed to better understand these . One possibility is that details of the ventral-stream architecture may be ill-suited to feedforward computation, simple processing units, and/or backpropagation, which could suggest differences between the way high-performance deep networks and the brain approach core object recognition. For most cortical areas, these are layers L2/3, L4, L5, and L6. For V1, these are L2/3(blob), L2/3(interblob), L4B, L4Cα, L4Cβ. For LGN, they are parvo, magno, and koniocellular divisions. reported validation accuracy of 79% on CIFAR-10, for a similar ventral-stream sub-network that 49 omitted connections with FLNe<0.15, and was trained for 50 epochs with the Adam update algorithm. In the present study, networks were trained for 300 epochs, using SGD with momentum 0.9, starting 51 with learning rate 0.1 and reducing it by 10x every 100 epochs. This ed in validation accuracy 52 of 84.59%. A standard DenseNet (github.com/kuangliu/pytorch-cifar) was trained using the same 53 procedure, ing in validation accuracy of 95.36%. To understand the basis of this performance gap, I created hybrid networks, with features of both the 55 ventral-stream network (VSN) and DenseNet. The VSN has a wide range of kernel sizes, optimized 56 to fill realistic receptive field sizes. In the first hybrid (H1), all kernel sizes of the VSN were set 57 to 3x3. The VSN also has a wide range of sparsity, with some connections consisting mostly of 58 zeros. In the second hybrid network (H2), in addition to using 3x3 kernels, I eliminated pixel-wise 59 sparsity, and limited channel-wise sparsity so that at least half of the input channels were used in 60 each connection. Thirdly (H3), I replaced each layer with a two-layer bottleneck module, specifically 61 a 1x1-kernel layer followed by 3x3 layer with four times fewer channels. The number of channels Most of these modifications improved performance, when applied cumulatively to make the ventral- stream increasingly similar to a DenseNet (Table 2,. In the monkey inferotemporal cortex data, mean selectivity is 3.5, and mean sparseness is 12.51. The ventral-stream model has much higher means, and DenseNet has much lower means. C. Representational dissimilarity, using a subset of images from. The plotted values are percentiles of one minus the Pearson correlations between responses to different stimuli. Monkey cell data shows relatively low values (high similarity) throughout the lower-right quarter of this matrix (spanning non-animal natural and artificial images), but neither of the deep networks does. efficiently, or the connection pattern of the ventral stream may be better suited to extracting certain 135 feature combinations in unsupervised learning than to communicating gradients through many layers. == An approximation of primate ventral stream as a convolutional network performs poorly on object recognition, and multiple architectural features contribute to this.  != We show shortcut connections should be placed in patterns that minimize between-layer distances during backpropagation, and design networks that achieve log L distances using L log(L) connections.

Question: We investigate methods for semi-supervised learning (SSL) of a neural linear-chain conditional random field (CRF) for Named Entity Recognition (NER) by treating the tagger as the amortized variational posterior in a generative model of text given tags. We first illustrate how to incorporate a CRF in a VAE, enabling end-to-end training on semi-supervised data. We then investigate a series of increasingly complex deep generative models of tokens given tags enabled by end-to-end optimization, comparing the proposed models against supervised and strong CRF SSL baselines on the Ontonotes5 NER dataset. We find that our best proposed model consistently improves performance by $\approx 1\%$ F1 in low- and moderate-resource regimes and easily addresses degenerate model behavior in a more difficult, partially supervised setting. Named entity recognition (NER) is a critical subtask of many domain-specific natural language understanding tasks in NLP, such as information extraction, entity linking, semantic parsing, and question answering. State-of-the-art models treat NER as a tagging problem (; ; ;), and while they have become quite accurate on benchmark datasets in recent years (; ; ; ;), utilizing them for new tasks is still expensive, requiring a large corpus of exhaustively annotated sentences . This problem has been largely addressed by extensive pretraining of high-capacity sentence encoders on massive-scale language modeling tasks;;; b), but it is natural to ask if we can squeeze more signal from our unlabeled data. Latent-variable generative models of sentences are a natural approach to this problem: by treating the tags for unlabeled data as latent variables, we can appeal to the principle of maximum marginal likelihood and learn a generative model on both labeled and unlabeled data. For models of practical interest, however, this presents multiple challenges: learning and prediction both require an intractable marginalization over the latent variables and the specification of the generative model can imply a posterior family that may not be as performant as the current state-of-the-art discriminative models. We address these challenges using a semi-supervised Variational Autoencoder (VAE) , treating a neural tagging CRF as the approximate posterior. We address the issue of optimization through discrete latent tag sequences by utilizing a differentiable relaxation of the Perturb-and-MAP algorithm (; ;), allowing for end-to-end optimization via backpropagation and SGD . Armed with this learning approach, we no longer need to restrict the generative model family (as in ;), and explore the use of rich deep generative models of text given tag sequences for improving NER performance. We also demonstrate how to use the VAE framework to learn in a realistic annotation scenario where we only observe a biased subset of the named entity tags. Our contributions can be summarized as follows: 1. We address the problem of semi-supervised learning (SSL) for NER by treating a neural CRF as the amortized approximate posterior in a discrete structured VAE. To the best of our knowledge, we are the first to utilize VAEs for NER. 2. We explore several variants of increasingly complex deep generative models of text given tags with the goal of improving tagging performance. We find that a joint tag-encoding Transformer architecture leads to an ≈ 1% improvement in F1 score over supervised and strong CRF SSL baselines. 3. We demonstrate that the proposed approach elegantly corrects for degenerate model performance in a more difficult partially supervised regime where sentences are not exhaustively annotated and again find improved performance. 4. Finally, we show the utility of our method in realistic low-and high-resource scenarios, varying the amount of unlabeled data. The ing high-resource model is competitive with state-of-the-art and, to the best of our knowledge, achieves the highest reported F1 score (88.4%) for models that do not use additional labeled data or gazetteers. We first introduce the tagging problem and tagging model. We then detail our proposed modeling framework and architectures. NER is the task of assigning coarsely-typed categories to contiguous spans of text. State-of-the-art approaches (; ; ; ; a) do so by treating span extraction as a tagging problem, which we now formally define. We are given a tokenized text sequence x 1:N ∈ X N and would like to predict the corresponding tag sequence y 1:N ∈ Y N which correctly encodes the observed token spans. 1 In this work, we use the BILOU tag-span encoding, which assigns four tags for each of the C span categories (e.g., B-PER, I-PER, L-PER, U-PER for the PERSON category.) The tag types B, I, L, U respectively encode beginning, inside, last, and unary tag positions in the original span. Additionally we have one O tag for tokens that are not in any named entity span. Thus our tag space has size |Y| = 4C + 1. We call the NER task of predicting tags for tokens inference, and model it with a discriminative distribution q φ (y 1:N |x 1:N) having parameters φ. Following state-of-the-art NER approaches (; ; ;), we use a neural encoding of the input followed by a linear-chain CRF decoding layer on top. We use the same architecture for q φ throughout this work, as follows: 1. Encode the token sequence, represented as byte-pairs, with a fixed pretrained language model. 2 That is, we first calculate: In our first experiments exploring the use of pretrained autoregressive information for generation (§3.1), we use the GPT2-SM model . In the experiments after (§3.2) we use the RoBERTa-LG model (b;). 2. Down-project the states: h. Combine local and transition potentials: ψ yi,yi+1 = s yi + T yi,yi+1, T yi,yi+1 ∈ R 5. Using special start and end states y 0 = *, y N +1 = with binary potentials ψ *,y = T *,y, ψ y, = T y, and the forward algorithm to compute the the partition function Z, we can compute the joint distribution: Our tagging CRF has trainable parameters φ = {W 1, b 1, V, b 2, T} 3 and we learn them on a dataset of fully annotated sentences D S = {(x i 1:N i, y i 1:N i)} using stochastic gradient descent (SGD) and maximum likelihood estimation. 2.3 SEMI-SUPERVISED CRF-VAE We now present the CRF-VAE, which treats the tagging CRF as the amortized approximate posterior in a Variational Autoencoder. We first describe our loss formulations for semi-supervised and partially supervised data. We then address optimizing these objectives end-to-end using backpropagation and the Relaxed Perturb-and-MAP algorithm. Finally, we propose a series of increasingly complex generative models to explore the potential of our modeling framework for improving tagging performance. The purpose of this work is to consider methods for estimation of q φ in semi-supervised data regimes, as in Kingma et al. This marginalization is intractable for models that are not factored among y i, so we resort to optimizing the familiar evidence lower bound (ELBO) with an approximate variational posterior distribution, which we set to our tagging model q φ. We maximize the ELBO on unlabeled data in addition to maximum likelihood losses for both the inference and generative models on labeled data, yielding the following objectives: where α is scalar hyper-parameter used to balance the supervised loss L S and the unsupervised loss L U . β is a scalar hyper-parameter used to balance the reconstruction and KL terms for the unsupervised loss . We note that, unlike a traditional VAE, this model contains no continuous latent variables. Assuming that supervised sentences are completely labeled is a restrictive setup for semi-supervised learning of a named entity tagger. It would be useful to be able to learn the tagger on sentences which are only partially labeled, where we only observe some named entity spans, but are not guaranteed all entity spans in the sentence are annotated and no O tags are manually annotated. 4 This presents a challenge in that we are no longer able to assume the usual implicit presence of O tags, since unannotated tokens are ambiguous. While it is possible to optimize the marginal likelihood of the CRF on only the observed tags y O, O ⊂ {1, . . ., N} in the sentence , doing so naively will in a degenerate model that never predicts O, by far the most common tag . Interestingly, this scenario is easily addressed by the variational framework via the KL term. We do this by reformulating the objective in Equation 5 to account for partially observed tag sequences: } be the partially observed dataset where, for some sentence i, O ⊂ {1, . . ., N i} is the set of observed positions and U = {1, . . ., N i} \ O is the set of unobserved positions. Our partially supervised objective is then which can be optimized as before using the constrained forward-backward and KL algorithms detailed in Appendix B. We also explore using this approach simply for regularization of the CRF posterior by omitting the token model p θ (x|y). Since we do not have trainable parameters for the generative model in this case, the reconstruction likelihood drops out of the objective and we have, for a single datum the following loss: Optimizing Equations 5 and 6 with respect to θ and φ using backpropagation and SGD is straightforward for every term except for the expectation terms E q φ (y|x) [log p θ (x|y)]. To optimize these expectations, we first make an Monte Carlo approximation using a single sample drawn from q φ. This discrete sample, however, is not differentiable with respect to φ and blocks gradient computation. While we may appeal to score function estimation (; ; ; ;) to work around this, its high-variance gradients make successful optimization difficult., we can compute approximate samples from q φ that are differentiable with respect to φ using the Relaxed Perturb-and-MAP algorithm . Due to space limitations, we leave the derivation of Relaxed Perturb-and-MAP for linear-chain CRFs to Appendix A and detail the ing CRF algorithms in Appendix B. We model the prior distribution of tag sequences y 1:N as the per-tag product of a fixed categorical distribution p(y 1:N) = i p(y i). The KL between q φ and this distribution can be computed in polynomial time using a modification of the forward recursion derived in , detailed in Appendix B. We experiment with several variations of architectures for p θ (x 1:N |y 1:N), presented in order of increasing complexity. The CRF Autoencoder is the previous state-of-the-art semi-supervised linear-chain CRF, which we consider a strong baseline. This model uses a tractable, fully factored generative model of tokens given tags and does not require approximate inference. Due to space limitations, we have detailed our implementation in Appendix C. MF: This is our simplest proposed generative model. We first embed the relaxed tag samples, represented as simplex vectors y i ∈ ∆ |Y|, into R dy p as the weighted combination of the input vector representations for each possible tag: We then compute factored token probabilities with an inner product where σ X is the softmax function normalized over X. This model is generalization of the CRF Autoencoder architecture in Appendix C where the tag-token parameters θ x,y are computed with a low-rank factorization W U. The restrictive factorization of MF is undesirable, since we expect that information about nearby tags may be discriminative of individual tokens. To test this, we extend MF to use the full tag context by encoding the embedded tag sequence jointly using a two-layer transformer with four attention heads per layer before predicting the tokens independently. That is, MF-GPT2: Next, we see if we can leverage information from a pretrained language model to provide additional training signal to p θ. We extend MF by adding the fixed pretrained language modeling parameters from GPT2 to the token scores: where z xi and h 0 i are the input token embeddings and hidden states from GPT2, respectively. We additionally normalize the scales of the factors by the square root of the vector dimensionalities to prevent the GPT2 scores from washing out the tag-encoding scores (d yp = 300 and d GP T 2 = 768). We add the same autoregressive extention to MT, using the tag encodings v instead of embeddings u. MT-GPT2-PoE: We also consider an autoregressive extension of MT, similar to MT-GPT2, that uses a product of experts (PoE) factorization instead MT-GPT2-Residual: Our last variation directly couples GPT2 with p θ by predicting a residual via a two-layer MLP based on the tag encoding and GPT2 state: For the MF-GPT2, MT-GPT2, and MT-GPT2-PoE models, we choose these factorizations specifically to prevent the trainable parameters from conditioning on previous word information, removing the possibility of the model learning to ignore the noisy latent tags in favor of the strong signal provided by pretrained encodings of the sentence histories . We further freeze the GPT2 parameters for all models, forcing the only path for improving the generative likelihood to be through the improved estimation and encoding of the tags y 1:N. We experiment first with the proposed models generative models for SSL and PSL in a moderately resourced regime (keeping 10% labeled data) to explore their relative merits. We then evaluate our best generative model from these experiments, (MT), with an improved bidirectional encoder language model in a low-and high-resource settings, varying the amount of unlabeled data. For data, we use the OntoNotes 5 NER corpus, which consists of 18 entity types annotated in 82,120 train, 12,678 validation, and 8,968 test sentences. We begin by comparing the proposed generative models, M* along with the following baselines: 1. Supervised (S): The supervised tagger trained only on the 10% labeled data. 2. Supervised 100% (S*): The supervised tagger trained on the 100% labeled data, used for quantifying the performance loss from using less data. 3. AE-Exact: The CRF Autoencoder using exact inference (detailed in Appendix C.) 4. AE-Approx: The same tag-token pair parameterization used by the CRF Autoencoder, but trained with the approximate ELBO objective as in Equation 11 instead of the exact objective in Equation 12. The purpose here is to see if we lose anything by resorting to the approximate ELBO objective. To simulate moderate-resource SSL, we keep annotations for only 10% of the sentences, yielding 8, 212 labeled sentences with 13, 025 annotated spans and 73, 908 unlabeled sentences. Results are shown in Table 1. All models except S* use this 10% labeled data. We first evaluate the proposed models and baselines without the use of a prior, since the use of a locally normalized factored prior can encourage overly uncertain joint distributions and degrade performance (; ;). We then explore the inclusion of the priors for the supervised and MT models with β = 0.01. We explore two varieties of prior tag distributions: the "gold" empirical tag distribution (Emp) from the full training dataset and a simple, but informative, hand-crafted prior (Sim) that places 50% mass on the O tag and distributes the rest of its mass evenly among the remaining tags. We view as a practical approach, since it does not require knowledge of the gold tag distribution, and use to quantify any relative disadvantage from not using the gold prior. We find that including the prior with a small weight, β = 0.01, marginally improved performance and interestingly, the simple prior outperforms the empirical prior, most likely because it is slightly smoother and does not emphasize the O tag as heavily. Curiously, we found that the approximate training of the CRF Autoencoder AE-Approx outperformed the exact approach AE-Exact by nearly 2% F1. We also note that our attempts to leverage signal from the pretrained autoregressive GPT2 states had negligible or negative effects on performance, thus we conclude that it is the addition of the joint encoding transformer architecture MT that provides the most gains (+0.8% F1). We also evaluate the supervised and transformer-based generative models, S and MT, on the more difficult PSL setup, where naively training the supervised model on the marginal likelihood of observed tags produces a degenerate model, due to the observation bias of never having O tags. In this setting we drop 90% of the annotations from sentences randomly, ing in 82,120 incompletely annotated sentences with 12,883 annotations total. We compare the gold and simple priors for each model. From the bottom of Table 1, we see that again our proposed transformer model MT outperforms the supervised-only model, this time by +1.3% F1. We also find that in this case, the MT models need to be trained with higher prior weights β = 0.1, otherwise they diverge towards using the O tag more uniformly with the other tags to achieve better generative likelihoods. 5 Code and experiments are available online at github.com/<anonymizedforsubmission> 6 In preliminary SSL experiments we found β > 0.01 to have a negative impact on performance, likely due to global/local normalization mismatch of the CRF and the prior. 7 The empirical prior puts 85% mass on the O tag Table 1: Semi-supervised and partially-supervised models on 10% supervised training data: best in bold, second best underlined. The proposed MT* improves performance in SSL and PSL by +1.1% F1 and +1.3% F1, respectively. Next we explore our best proposed architecture MT and the supervised baseline in low-and highresource settings (1% and 100% training data, respectively) and study the effects of training with an additional 100K unlabeled sentences sampled from Wikipedia (detailed in Appendix E). Since we found no advantage from using pretrained GPT2 information in the previous experiment, we evaluate the use of the bidirectional pretrained language model, RoBERTa (b), since we expect bidirectional information to highly benefit performance (; , among others). We also experiment with a higher-capacity tagging model, S-LG, by adding more trainable Transformers (L = 4, A = 8, H = 1024) between the RoBERTa encodings and down-projection layers. From Table 2 we see that, like in the 10% labeled data setting, the CRF-VAE improves upon the supervised model by 0.9% F1 in this 1% setting, but we find that including additional data from Wikipedia has a negative impact. A likely reason for this is the domain mismatch between Ontonotes5 and Wikipedia (news and encyclopedia, respectively). In the high-resource setting, we find that using RoBERTa significantly improves upon GPT2 (+5.7% F1) and the additional capacity of S-LG further improves performance by +2.2% F1. Although we do not see a significant improvement from semi-supervised training with Wikipedia sentences, our model is competitive with previous state-of-the-art NER approaches and outperforms all previous approaches that do not use additional labeled data or gazetteers. Utilizing unlabeled data for semi-supervised learning in NER has been studied considerably in the literature. A common approach is a two-stage process where useful features are learned from unsupervised data, then incorporated into models which are then trained only on the supervised data . With the rise of neural approaches, large-scale word vector and language model pretraining methods; ) can be regarded in the same vein. Table 2: Low-and high-resource with RoBERTa, varying available unlabeled data. Best scores not using additional labeled data in bold. † Uses additional labeled data or gazetteers. Another approach is to automatically create silver-labeled data using outside resources, whose low recall induces a partially supervised learning problem. approach the problem by distantly supervising spans using a database. similarly use a gazetteer and adapt the structured perceptron to handle partially labeled sequences, while optimize the marginal likelihood of the distantly annotated tags.'s method, however, still requires some fully labeled data to handle proper prediction of the O tag. The problem setup from is the same as our PSL regime, but they use a cross-validated self-training approach. use a marginal likelihood objective to pool overlapping NER tasks and datasets, but must exploit datasetspecific constraints, limiting the allowable latent tags to debias the model from never predicting O tags. Generative latent-variable approaches also provide an attractive approach to learning on unsupervised data. present an approach that uses the CRF for autoencoding and extend it to neural CRFs, but both require the use of a restricted factored generative model to make learning tractable. Deep generative models of text have shown promise in recent years, with demonstrated applications to document representation learning, sentence generation , compression, translation , and parsing . However, to the best of our knowledge, this framework has yet to be utilized for NER and tagging CRFs. A key challenge for learning VAEs with discrete latent variables is optimization with respect to the inference model parameters φ. While we may appeal to score function estimation (; ; ;, its empirical high-variance gradients make successful optimization difficult. Alternatively, obtaining gradients with respect to φ can be achieved using the relaxed Gumbel-max trick and has been recently extended to latent tree-CRFs by , which we make use of here for sequence CRFs. We proposed a novel generative model for semi-supervised learning in NER. By treating a neural CRF as the amortized variational posterior in the generative model and taking relaxed differentiable samples, we were able to utilize a transformer architecture in the generative model to condition on more context and provide appreciable performance gains over supervised and strong baselines on both semi-supervised and partially-supervised datasets. We also found that inclusion of powerful pretrained autoregressive language modeling states had neglible or negative effects while using a pretrained bidirectional encoder offers significant performance gains. Future work includes the use of larger in-domain unlabeled corpora and the inclusion of latent-variable CRFs in more interesting joint semi-supervised models of annotations, such as relation extraction and entity linking. ) and τ ≥ 0 be the temperature: We know from that the MAP sequence from this perturbed distribution is a sample from the unperturbed distribution. Coupled with the property that the zero temperature limit of the Gibbs distribution is the MAP state , it immediately follows that the zero temperature limit of the perturbedq is a sample from q: ⇒ lim τ →0q where q φ (y|x; τ) is the tempered but unperturbed q φ and "one-hot" is a function that converts elements of Y N to a one-hot vector representation. Thus we can use the temperature τ to anneal the perturbed joint distributionq φ (y|x; τ) to a sample from the unperturbed distribution,ỹ ∼ q φ. When τ > 0,q φ (y|x; τ) is differentiable and can be used for end-to-end optimization by allowing us to approximate the expectation with a relaxed single-sample Monte Carlo estimate: where we have modified log p θ (x|y) to accept the simplex representations of y 1:N fromq φ instead of discrete elements, which has the effect of log p θ (x|y) computing a weighted combination of its input vector representations for y ∈ Y similarly to an attention mechanism or the annotation function in (see Equation 7.) This can be thought of as a generalization of the Gumbel-softmax trick from; to structured joint distributions. The statements in also imply something of practical interest: we can compute the argmax (Viterbi decoding) and its differentiable relaxation; a sample and its differentiable relaxation; the partition function; and the marginal tag distributions, all using the same sum-product algorithm implementation, controlled by the temperature and the presence of noise. We have detailed the algorithm in Appendix B. In Algorithm 1 we have detailed the stable, log-space implementation of the generalized forwardbackward algorithm for computing the argmax (Viterbi decoding) and its differentiable relaxation; a sample and its differentiable relaxation; the partition function; and the marginal tag distributions below. While this algorithm does provide practical convenience, we note that real implementations should have separate routines for computing the partition function (running only the forward algorithm), and the discrete τ = 0 Viterbi algorithm, since it is more numerically stable and efficient. We also have included the dynamic program for computing the constrained KL divergence between q φ and a factored p(y) in Algorithm 2. The idea of using a CRF to reconstruct tokens given tags for SSL has been explored before by; , which we consider to be a strong baseline and restate ψ yi,yi+1 + log p θ (x i |y i)} − log Z(ψ) = log Z(ψ + log p θ) − log Z(ψ) where log Z(ψ + log p θ) is a slight abuse of notation intended to illustrate that the first term in Equation 12 is the same computation as the partition function, but with the generative log-likelihoods added to the CRF potentials. We note that instead of using the Mixed-EM procedure from , we model p θ (x i |y i) using free logit parameters θ x,y for each token-tag pair and normalize using a softmax, which allows for end-to-end optimization via backpropagation and SGD. We train each model to convergence using early-stopping on the F1 score of the validation data, with a patience of 10 epochs. For all models that do not have trainable transformers, we train using the Adam optimizer with a learning rate of 0.001, and a batch size of 128. For those with transformers (MT*), we train using Adam, a batch size of 32, and the Noam learning rate schedule from with a model size of d yp = 300 and 16, 000 warm-up steps . Additionally, we use gradient clipping of 5 for all models and a temperature of τ =.66 for all relaxed sampling models. We implemented our models in PyTorch using the AllenNLP framework and the implementation of the pretrained GPT2 and RoBERTa. We have made all code, data, and experiments available online at github.com/ <anonymizedforsubmission> for reproducibility and reuse. All experimental settings can be reproduced using the configuration files in the repo. For the experiments in §3.2, we gather an additional training corpus of out-of-domain encyclopedic sentences from Wikipedia. To try to get a sample that better aligns with the Ontonotes5 data, these sentences were gathered with an informed process, which was performed as follows: 1. Using the repository <anonymized for submission>, we extract English Wikipedia and align it with Wikidata. 2. We then look up the entity classes from the Ontonotes5 specification in Wikidata and, for each NER class, find all Wikidata classes that are below this class in ontology (all subclasses). 3. We then find all items which are instances of these classes and also have Wikipedia pages. These are the Wikipedia entities which are likely to be instances of the NER classes.

Represent this text to find a much much longer article about it with say hundreds-thousands of words.

We embed a CRF in a VAE of tokens and NER tags for semi-supervised learning and show improvements in low-resource settings.

We learn an unsupervised learning algorithm that produces useful representations from a set of supervised tasks. At test-time, we apply this algorithm to new tasks without any supervision and show performance comparable to a VAE.


Question: Represent the scientific article to find a short TLDR!

Question: To make deep neural networks feasible in resource-constrained environments (such as mobile devices), it is beneficial to quantize models by using low-precision weights. One common technique for quantizing neural networks is the straight-through gradient method, which enables back-propagation through the quantization mapping. Despite its empirical success, little is understood about why the straight-through gradient method works. Building upon a novel observation that the straight-through gradient method is in fact identical to the well-known Nesterov’s dual-averaging algorithm on a quantization constrained optimization problem, we propose a more principled alternative approach, called ProxQuant, that formulates quantized network training as a regularized learning problem instead and optimizes it via the prox-gradient method. ProxQuant does back-propagation on the underlying full-precision vector and applies an efficient prox-operator in between stochastic gradient steps to encourage quantizedness. For quantizing ResNets and LSTMs, ProxQuant outperforms state-of-the-art on binary quantization and is on par with state-of-the-art on multi-bit quantization. For binary quantization, our analysis shows both theoretically and experimentally that ProxQuant is more stable than the straight-through gradient method (i.e. BinaryConnect), challenging the indispensability of the straight-through gradient method and providing a powerful alternative. change metric. We present the main ingredients of our contribution in this extended abstract. See the Appendices B for the prior work, C for the notation, A and D for the motivation and preliminary discussions about 50 the straight-through gradient method and prox operators. 2 Quantized net training via regularized learning 52 We propose the PROXQUANT algorithm, which adds a quantization-inducing regularizer onto the 53 loss and optimizes via the (non-lazy) prox-gradient method with a finite λ. The prototypical version 54 of PROXQUANT is described in Algorithm 1. Require: Regularizer R that induces desired quantizedness, initialization θ 0, learning rates {η t} t≥0, regularization strengths {λ t} t≥0 while not converged do Perform the prox-gradient step DISPLAYFORM0 = prox ηtλtR θ t − η t ∇L(θ t).The inner SGD step in eq. can be replaced by any preferred stochastic optimization method such as Momentum SGD or Adam []. end whileCompared to usual full-precision training, PROXQUANT only adds a prox step after each stochastic Table 1: Top-1 classification error of quantized ResNets on CIFAR-10. Performance is reported in mean(std) over 4 runs, where for PQ-T we report in addition the best of 4 (Bo4). We now show that BinaryConnect has a very stringent convergence condition. Consider the Bina-to find a parameter θ ∈ {±1} d with low loss, the algorithm only has access to stochastic gradients at {±1} d. As this is a discrete set, a priori, gradients in this set do not necessarily contain any (a E H 9 I S e j X v j 0 X g x X h e l K 8 a y 5 w j 9 g P H 2 C a M i n A U = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " DISPLAYFORM0 a E H 9 I S e j X v j 0 X g x X h e l K 8 a y 5 w j 9 g P H 2 C a M i n A U = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " DISPLAYFORM1 a E H 9 I S e j X v j 0 X g x X h e l K 8 a y 5 w j 9 g P H 2 C a M i n A U = < / l a t e x i t > < l a t e x i t s h a 1 _ b a s e 6 4 = " DISPLAYFORM2 a E H 9 I S e j X v j 0 X g x X h e l K 8 a y 5 w j 9 g P H 2 C a M i n A U = < / l a t e x i t > rL(✓t) < l a t e x i t s h a 1 _ b a s e 6 4 = " k M y 6 L o 8 X P y Figure 1: (a) Comparison of the straight-through gradient method and our PROXQUANT method. The straightthrough method computes the gradient at the quantized vector and performs the update at the original real vector; PROXQUANT performs a gradient update at the current real vector followed by a prox step which encourages quantizedness. (b) A two-function toy failure case for BinaryConnect. The two functions are f1(x) = |x + 0.5| − 0.5 (blue) and f−1(x) = |x − 0.5| − 0.5 (orange). The derivatives of f1 and f−1 coincide at {−1, 1}, so any algorithm that only uses this information will have identical behaviors on these two functions. However, the minimizers in {±1} are x 1 = −1 and x −1 = 1, so the algorithm must fail on one of them. DISPLAYFORM3 sparsification, nearest-neighbor clustering, and Huffman coding. This architecture is then made into 173 a specially designed hardware for efficient inference []. In a parallel line of work, parameter, but might be hard to tune due to the instability of the inner maximization problem. While preparing this manuscript, we discovered the independent work of Carreira-Perpinán, Carreira-Perpinán and Idelbayev. They formulate quantized network training as a constrained DISPLAYFORM0 This enables the real vector θ to move in the entire Euclidean space, and taking q(θ) at the end of training gives a valid quantized model. Such a customized back-propagation rule yields good empiri-227 cal performance in training quantized nets and has thus become a standard practice [Courbariaux 228 et al., 2015, , . However, as we have discussed, it is information 229 theoretically unclear how the straight-through method works, and it does fail on very simple convex Lipschitz functions (Figure 1b). D.2 Straight-through gradient as lazy projection Our first observation is that the straight-through gradient method is equivalent to Nesterov's dual- averaging method, or a lazy projected SGD []. In the binary case, we wish to minimize 234 L(θ) over Q = {±1} d, and the lazy projected SGD proceeds as DISPLAYFORM0 Written compactly, this is θ t+1 = θ t −η t ∇L(θ)| θ=q(θt), which is exactly the straight-through gradient 236 method: take the gradient at the quantized vector and perform the update on the original real vector. We take a broader point of view that a projection is also a limiting proximal operator with a suitable 239 regularizer, to allow more generality and to motivate our proposed algorithm. Given any set Q, one 240 could identify a regularizer R: R d → R ≥0 such that the following hold: DISPLAYFORM0 In the case Q = {±1} d for example, one could take DISPLAYFORM1 The proximal operator (or prox operator) [] with respect to R and strength DISPLAYFORM2 In the limiting case λ = ∞, the argmin has to satisfy R(θ) = 0, i.e. θ ∈ Q, and the prox operator is 245 to minimize θ − θ 0 2 2 over θ ∈ Q, which is the Euclidean projection onto Q. Hence, projection is 246 also a prox operator with λ = ∞, and the straight-through gradient estimate is equivalent to a lazy 247 proximal gradient descent with and λ = ∞. While the prox operator with λ = ∞ correponds to "hard" projection onto the discrete set Q, when 249 λ < ∞ it becomes a "soft" projection that moves towards Q. Compared with the hard projection, 250 a finite λ is less aggressive and has the potential advantage of avoiding overshoot early in training. Further, as the prox operator does not strictly enforce quantizedness, it is in principle able to query 252 the gradients at every point in the space, and therefore has access to more information than the 253 straight-through gradient method. E Details on the PROXQUANT algorithm 255 E.1 Regularization for model quantization We define a flexible class of quantization-inducing regularizers through "distance to the quantized 257 set", derive efficient algorithms of their corresponding prox operator, and propose a homotopy method 258 for choosing the regularization strengths. Our regularization perspective subsumes most existing 259 algorithms for model-quantization (e.g., [, , 260 as limits of certain regularizers with strength λ → ∞. Our proposed method can be viewed as a 261 principled generalization of these methods to λ < ∞. Let Q ⊂ R d be a set of quantized parameter vectors. An ideal regularizer for quantization would be 263 to vanish on Q and reflect some type of distance to Q when θ / ∈ Q. To achieve this, we propose L 1 and L 2 regularizers of the form DISPLAYFORM0 This is a highly flexible framework for designing regularizers, as one could specify any Q and choose between L 1 and L 2 . Specifically, Q encodes certain desired quantization structure. By appropriately 267 choosing Q, we can specify which part of the parameter vector to quantize 1, the number of bits to 268 quantize to, whether we allow adaptively-chosen quantization levels and so on. The choice of distance metrics will in distinct properties in the regularized solutions. For 270 example, choosing the L 1 version leads to non-smooth regularizers that induce exact quantizedness 271 in the same way that L 1 norm regularization induces sparsity [], whereas choosing 272 the squared L 2 version leads to smooth regularizers that induce quantizedness "softly". In the following, we present a few examples of regularizers under our framework eq. which induce 274 binary weights, ternary weights and multi-bit quantization. We will also derive efficient algorithms (or approximation heuristics) for solving the prox operators corresponding to these regularizers, which generalize the projection operators used in the straight-through gradient algorithms. Binary neural nets In a binary neural net, the entries of θ are in {±1}. A natural choice would be DISPLAYFORM0 This is exactly the binary regularizer R bin that we discussed earlier in eq.. FIG4 plots the 280 W-shaped one-dimensional component of R bin from which we see its effect for inducing {±1} 281 quantization in analog to L 1 regularization for inducing exact sparsity. The prox operator with respect to R bin, despite being a non-convex 283 optimization problem, admits a simple analytical solution: DISPLAYFORM1 DISPLAYFORM2 1 Empirically, it is advantageous to keep the biases of each layers and the BatchNorm layers at full-precision, which is often a negligible fraction, say 1/ √ d of the total number of parameters alternating quantizer of []: Bα = q alt (θ). Together, the prox operator generalizes the alternating minimization procedure in [], as 300 λ governs a trade-off between quantization and closeness to θ. To see that this is a strict generalization, note that for any λ the solution of eq. will be an interpolation between the input θ and its Euclidean 302 projection to Q. As λ → +∞, the prox operator collapses to the projection. Ternary quantization Ternary quantization is a variant of 2-bit quantization, in which weights are 304 constrained to be in {−α, 0, β} for real values α, β > 0. For ternary quantization, we use an approximate version of the alternating prox operator eq. FORMULA0: DISPLAYFORM0 by initializing at θ = θ and repeating DISPLAYFORM1 where q is the ternary quantizer defined as DISPLAYFORM2 This is a straightforward extension of the TWN quantizer [] Recall that the larger λ t is, the more aggressive θ t+1 will move towards the quantized set. An ideal 313 choice would be to force the net to be exactly quantized upon convergence, and not be too 314 aggressive such that the quantized net at convergence is sub-optimal. We let λ t be a linearly increasing sequence, i.e. λ t:= λ · t for some hyper-parameter λ > 0 which 316 we term as the regularization rate. With this choice, the stochastic gradient steps will start off 317 close to full-precision training and gradually move towards exact quantizedness, hence the name 318 "homotopy method". The parameter λ can be tuned by minimizing the validation loss, and controls 319 the aggressiveness of falling onto the quantization constraint. There is nothing special about the 320 linear increasing scheme, but it is simple enough and works well as we shall see in the experiments. Problem setup We perform language modeling with LSTMs Hochreiter and Schmidhuber 323 on the Penn Treebank (PTB) dataset [], which contains 929K training tokens, 73K validation tokens, and 82K test tokens. Our model is a standard one-hidden-layer LSTM with 325 embedding dimension 300 and hidden dimension 300. We train quantized LSTMs with the encoder, 326 transition matrix, and the decoder quantized to k-bits for k ∈ {1, 2, 3}. The quantization is performed 327 in a row-wise fashion, so that each row of the matrix has its own codebook {α 1, . . ., α k}. multi-bit quantization, we also report the for binary LSTMs (weights in {±1}), comparing BinaryConnect and our PROXQUANT-Binary. Result We report the perplexity-per-word (PPW, lower is better) in TAB3. The performance 337 of PROXQUANT is comparable with the Straight-through gradient method. On Binary LSTMs, PROXQUANT-Binary beats BinaryConnect by a large margin. These demonstrate that PROX- QUANT offers a powerful alternative for training recurrent networks. We experimentally compare the training dynamics of PROXQUANT-Binary and BinaryConnect In R d, the space of all full-precision parameters, the sign change is a natural distance metric that 345 represents the closeness of the binarization of two parameters. Recall in our CIFAR-10 experiments (Section 3.1), for both BinaryConnect and PROXQUANT, we 347 initialize at a good full-precision net θ 0 and stop at a converged binary network θ ∈ {±1} d. We 348 are interested in SignChange(θ 0, θ t) along the training path, as well as SignChange(θ 0, θ), i.e. the 349 distance of the final output model to the initialization. As PROXQUANT converges to higher-performance solutions than BinaryConnect, we expect that if we run both methods from a same warm start, the sign change of PROXQUANT should be higher than 352 that of BinaryConnect, as in general one needs to travel farther to find a better net. However, we find that this is not the case: PROXQUANT produces binary nets with both lower sign BinaryConnect never stop changing until we manually freeze the signs at epoch 400. G.1 Detailed sign change on ResNet-20 362 2 We thank Xu et al. for sharing the implementation of this method through a personal communication. There is a very clever trick not mentioned in their paper: after computing the alternating quantization q alt (θ), they multiply by a constant 0.3 before taking the gradient; in other words, their quantizer is a rescaled alternating quantizer: θ → 0.3q alt (θ). This scaling step gives a significant gain in performance -without scaling the PPW is {116.7, 94.3, 87.3} for {1, 2, 3} bits. In contrast, our PROXQUANT does not involve a scaling step and achieves better PPW than this unscaled ALT straight-through method. BC 9.664, 9.430, 9.198, 9.663 0.386, 0.377, 0.390, 0.381 (8.06) PQ-B 9. 058, 8.901, 9.388, 9.237 0.288, 0.247, 0.284, 9.530, 9.623, 10.370 0.376, 0.379, 0.382, 0.386 (8.31) 9.474, 9.410, 9.370 0.291, 0.287, 0.289, 9.558, 9.538, 9.328 0.360, 0.357, 0.359, 0.360 (7.73) PQ-B 9. 284, 8.866, 9.301, 8.884 0.275, 0.276, 0.276, 0.275

Represent this text to find a much much longer article about it with say hundreds-thousands of words

A principled framework for model quantization using the proximal gradient method.

Represent the TLDR for finding the corresponding scientific article

We show shortcut connections should be placed in patterns that minimize between-layer distances during backpropagation, and design networks that achieve log L distances using L log(L) connections.


Question: Represent the article for finding a single-sentence TLDR

Question: The vulnerabilities of deep neural networks against adversarial examples have become a significant concern for deploying these models in sensitive domains. Devising a definitive defense against such attacks is proven to be challenging, and the methods relying on detecting adversarial samples are only valid when the attacker is oblivious to the detection mechanism. In this paper, we consider the adversarial detection problem under the robust optimization framework. We partition the input space into subspaces and train adversarial robust subspace detectors using asymmetrical adversarial training (AAT). The integration of the classifier and detectors presents a detection mechanism that provides a performance guarantee to the adversary it considered. We demonstrate that AAT promotes the learning of class-conditional distributions, which further gives rise to generative detection/classification approaches that are both robust and more interpretable. We provide comprehensive evaluations of the above methods, and demonstrate their competitive performances and compelling properties on adversarial detection and robust classification problems. Deep neural networks have become the staple of modern machine learning pipelines, achieving stateof-the-art performance on extremely difficult tasks in various applications such as computer vision , speech recognition , machine translation , robotics , and biomedical image analysis . Despite their outstanding performance, these networks are shown to be vulnerable against various types of adversarial attacks, including evasion attacks (aka, inference or perturbation attacks) (; b; b;) and poisoning attacks . These vulnerabilities in deep neural networks hinder their deployment in sensitive domains including, but not limited to, health care, finances, autonomous driving, and defense-related applications and have become a major security concern. Due to the mentioned vulnerabilities, there has been a recent surge toward designing defense mechanisms against adversarial attacks (; ; b; ; ;), which has in turn motivated the design of stronger attacks that defeat the proposed defenses (b; b; a; b; ; ; ;). Besides, the proposed defenses have been shown to be limited and often not effective and easy to overcome . Alternatively, a large body of work has focused on detection of adversarial examples (; ; ; ; ; ; ; ; ; ; ; ; ;). While training robust classifiers focuses on maintaining performance in presence of adversarial examples, adversarial detection only cares for detecting these examples. The majority of the current detection mechanisms focus on non-adaptive threats, for which the attacks are not specifically tuned/tailored to bypass the detection mechanism, and the attacker is oblivious to the detection mechanism. In fact, Carlini & Wagner (2017a) and showed that the detection methods presented in (; ; ; ; ; ; ;), are significantly less effective than their claimed performances under adaptive attacks. The current solutions are mostly heuristic approaches that cannot provide performance guarantees to the adversary they considered. In this paper, we are interested in detection mechanisms for adversarial examples that can withstand adaptive attacks. Unlike previous approaches that assume adversarial and natural samples coming from different distributions, thus rely on using a single classifier to distinguish between them, we instead partition the input space into subspaces based on the classification system's output and perform adversarial/natural sample classification in these subspaces. Importantly, the mentioned partitions allow us to drop the adversarial constrain and employ a novel asymmetrical adversarial training (AAT) objective to train robust binary classifiers in the subspaces. Figure 1 demonstrates our idea of space partitioning and robust detector training. Our qualitative show that AAT supports detectors to learn class-conditional distributions, which further motivates generative detection/classification solutions that are both robust and interpretable. Our specific contributions are: • We develop adversarial example detection techniques that provide performance guarantees to norm constrained adversaries. Empirically, our best models improve previous state-ofthe-art mean L 2 distortion from 3.68 to 4.47 on the MNIST dataset, and from 1.1 to 1.5 on the CIFAR10 dataset. • We study powerful and versatile generative classification models derived from our detection framework and demonstrate their competitive performances over discriminative robust classifiers. While defense mechanisms based on ordinary adversarial training are vulnerable to unrecognizable inputs (e.g., rubbish examples), inputs that cause confident predictions of our models have human-understandable semantic meanings. • We demonstrate that AAT not only induces robustness as ordinary adversarial training methods do, but also promotes the learning of class-conditional distributions. Intuitively, the learning mechanism is similar to that of GANs, but the objective doesn't learn a fixed generator. On 1D and 2D benchmarking datasets we show this flexibility allows us to precisely control the data generation process such that the detector could be pushed to a good approximation of the underlying density function. (In case of GANs at the global optimum the discriminator converges to a degenerated uniform solution.) Our image generation on CIFAR10 and ImageNet rival that of state-of-the-art GANs. Adversarial attacks. Since the pioneering work of , a large body of work has focused on designing algorithms that achieve successful attacks on neural networks (b; ; b; ; a; b). More recently, iterative projected gradient descent (PGD), initially proposed by Kurakin et al. (2016b), has been empirically identified as the most effective approach for performing norm ball constrained attacks, and the attack reasonably approximates the optimal attack . Adversarial detection techniques. The majority of the methods developed for detecting adversarial attacks are based on the following core idea: given a trained K-class classifier, f: R d → {1...K}, and its corresponding natural training samples,, generate a set of adversarially attacked samples, and devise a mechanism to discriminate D from D. For instance, use this exact idea and learn a binary classifier to distinguish the natural and adversarially perturbed sets. append a new "attacked" class to the classifier, f, and re-train a secured network that classifies natural images, x ∈ D, into the K classes and all attacked images, x ∈ D, to the (K + 1)-th class. In contrast to; , which aim at detecting adversarial examples directly from the image content, trained a binary classifier that receives as input the intermediate layer features extracted from the classifier network f, and distinguished D from D based on such input features. More importantly, considered the so-called case of adaptive/dynamic adversary and proposed to harden the detector against such attacks using a similar adversarial training approach as in Goodfellow et al. (2014b). Unfortunately, the mentioned detection methods are significantly less effective under an adaptive adversary equipped with a strong attack (a;)... K} be the classifier that is used to do classification on D. With the labels and predicted labels the dataset respectively forms the partition, 1} be a set of binary classifiers (detectors), in which h k is trained to discriminate natural samples classified as k, from adversarial samples that fool the network, f (·), to be classified as k. Also, let D be a set of L p norm bounded adversarial examples crafted from D: Consider the following procedure to determine whether a sample x in D ∪ D is an adversarial example: First obtain the estimated class label k:= f (x), then use the k-th detector to predict: if h k (x) = 1 then x a natural sample, otherwise it's an adversarial sample. The detection accuracy of the algorithm is given by where Thus minimizing the algorithm's classification error is equivalent to minimizing classification error of individual detectors. Employing empirical risk minimization, detector k, parameterized by θ k, is trained by where L is a loss function that measures the distance between h k's output and the supplied label (e.g., the binary cross-entropy loss). In the case of adaptive attacks, when the adversary aims to fool both the classifier and detectors, the accuracy of a naively trained detector could be significantly reduced. In order to be robust to adaptive attacks, inspired by the idea of robust optimization , we incorporate the attack into the training objective: where D f \k = {x : f (x) = k, y = k, x ∈ D}, and we assume that perturbation budget is large enough such that ∀x ∈ D f \k, ∃δ ∈ S, s.t. f (x + δ) = k. Now by dropping the f (x + δ) = k constrain we could derive an upper bound for the first loss term: max The detector could instead be trained by minimizing this upper bound using the following unconstrained objective, Further, we use the fact that when D is used as the training set, f could overfit on D such that D \k = {x i : y i = k} and D k are respectively good approximations of D f \k and D f k. This leads to our proposed asymmetrical adversarial training (AAT) objective: In a nutshell, each detector is trained using in-class natural samples and detector-adversarial examples crafted from out-of-class samples. We use iterative PGD attack to solve the inner maximization. Because of the integrated adversary, objective 5 is no longer a straightforward discriminative objective. Our investigations (Appendix A) showed that the objective promotes detectors to learn conditional data distributions. Similar to GANs' objective (a), the AAT objective presents a minimax problem, where the adversary tries to generate perturbed samples that look like the target class data, and the detector is trained by discriminating between target class data and perturbed data. The key difference is that instead of learning a mapping from latent space distribution to the data distribution (a.k.a., the generator), AAT relies on gradient descent (a.k.a., PGD attack) to generate data samples. This is crucial, as it allows us to perform fine-grained control over the generation process (especially by constraining on perturbation limit), so that the discriminator (detector) could retain density information (see Appendix A) and not converge to a degenerate uniform solution as in the case of GANs (a;). Unfortunately, the detector does not define an explicit density function. Under the energy-based learning framework , we could, however, obtain the joint probability of the input and a class category using the Gibbs distribution:, where Z Θ is an unknown normalizing constant, and E θ k (x) = −z(h k (x)) (see Appendix A for justification). We could then apply the Bayes classification rule to obtain a generative classifier: H(x) = arg max k p(x, k) = arg max k z(h k (x)). In addition, we could base on p(x, k) to reject low probability inputs. We implement the reject option by thresholdingk-th detector's logit output, wherek is the predicted class. In the context of adversarial example detection, rejected samples are considered as adversarial examples. We first test the robustness of individual detectors. We show that, once we train a detector with an adequately configured PGD attack, its performance cannot be significantly reduced by an adversary with much stronger configurations (stronger in terms of steps and step-size). Although the PGD attack can reasonably solve the inner maximization in objective 5, it is not clear whether the optimization landscape of the asymmetrical objective is the same as its symmetrical counterparts. For instance, we found that the step-size used by to train their CIFAR10 robust classifier would not induce robustness to our detectors (see Appendix D.2.2). We also face a unique challenge when training with objective 5: the number of positive and negative samples are highly imbalanced. Our solution is to use re-sampling to balance positive and negative classes. Furthermore, we use adversarial finetuning on CIFAR10 and ImageNet to speed up the training of our detectors. With the robustness test, we show that robust optimization also introduces robustness within this new training paradigm. We use AUC (area under the ROC Curve) to measure detection performances. The metric could be interpreted as the probability that the detector assigns a higher score to a random positive sample than to a random negative example. While the true-positive and the false-positive rates are the commonly used metrics for measuring the detection performance, they require a detection threshold to be specified. AUC, however, is an aggregated measurement of detection performance across a range of thresholds, and we found it to be a more stable and reliable metric. For the k-th detector h k, its AUC is computed on the set {(x, 0): \k )} (refer to loss 4). Having validated the robustness of individual detectors, we evaluate the overall performance of our integrated detection system. Recalling our detection rule, we first obtain the estimated class label k:= f (x), then use the k-th detector's logit output z(h k (x)) to predict: if z(h k (x)) ≥ T k, then x is a natural sample, otherwise it is an adversarially perturbed sample. For the sake of this evaluation, we use a universal threshold for all the detectors: ∀k ∈ {1...K} T k = T, and report detection performance at a range of universal thresholds. In practice, however, the optimal value of each detector's detection threshold T k should be determined by optimizing a utility function. to denote the test set that contains natural samples, and to denote the corresponding perturbed test set. For a given threshold T, we compute the true positive rate (TPR) on D and false positive rate (FPR) on D. These two metrics are respectively defined as and In the FPR definition we use f (x) = y to constrain that only true adversarial examples are counted as false positives. This constraint is necessary, as we found that for the norm ball constraint we considered in the experiments, not all perturbed samples are adversarial examples that cause misclassification on f. In order to craft the perturbed dataset D, we consider three attacking scenarios. Classifier attack. This attack corresponds to the scenario where the adversary is oblivious to the detection mechanism. For a given natural sample x and its label y, the perturbed sample x is computed by minimizing the loss, where z(f (x)) is the classifier's logit outputs. This objective is derived from the CW attack (b) and used in MadryLab (b) and MadryLab (a) to perform untargeted attacks. Detectors attack. In this scenario adversarial examples are produced by attacking only the detectors. We construct a single detection function H by using the i-th detector's logit output as its i-th logit output:. H is then treated as a single network, and the perturbed sample x for a given input (x, y) is computed by minimizing the loss Note that, according to our detection rule, a low value of the detector's logit output indicates detection of an adversarial example, thus by minimizing the negative of logit output we make the perturbed example harder to detect. H could also be fed directly to the CW loss 8 or to cross-entropy loss, but we found the attack based on the loss in 9 to be significantly more effective. Combined attack. With the goal of fooling both the classifier and detectors, perturbed samples are produced by attacking the integrated detection system. We consider two loss functions for realizing the combined attack. The first is based on the combined loss function (a) that has been shown to be effective against an array of detection methods. Given a natural example x and its label y, same as the detectors-attack scenario, we first construct a single detection function H by aggregating the logit outputs of individual detectors: z(H(x)) i:= z(h i (x)). We then use the aggregated detector's largest logit output max k =y z(H(x)) k (low value of this quantity indicates detection of an adversarial example) and the classifier logit outputs z(f (x)) to construct a surrogate classifier g, with its logit outputs being A perturbed example x is then computed by minimizing the loss function In practice we observe that the optimization of this loss tends to stuck at the point where max i =y z(f (x)) i keeps changing signs while max j =y z(H(x)) j stays as a large negative number (which indicates detection). To derive a more effective attack we consider a simple combination of loss 8 and loss 9: The objective is straightforward: if x is not yet an adversarial example on f, optimize it for that goal; otherwise optimize it for fooling the aggregated detector. We mention briefly here that we perform the same performance analysis of our generative detection method (as detailed in Section 3.2) by computing TPR on D and FPR on D. We use the loss 9 to perform attacks against the generative detection method, but also provide of attacks based on cross-entropy loss and CW loss 8. Integrated classification. In addition to the generative classifier proposed in Section 3.2, we introduce another classification scheme that provides a reject option. The scheme is based an integration of the naive classifier f and the detectors: for a given input x and its prediction label We respectively use loss 12 and 9 to attack the integrated classifier and the generative classifier. Performance metric. In the context of robust classification, the performance of a robust classifier is measured using standard accuracy and robust accuracy -accuracies respectively computed on the natural dataset and perturbed dataset. We provide a similar performance analysis of the above classification models. On the natural dataset, we compute the accuracy as the fraction of samples that are correctly classified (f (x) = y) and at the same time not rejected (z(h k (x)) ≥ T ): On the perturbed dataset we compute the error as the fraction of samples that are misclassified (f (x) = y) and at the same time not rejected: Note that in this case the error is no longer a complement of the accuracy. For a classification system with a reject option, any perturbed samples that are rejected should be considered as properly handled, regardless of whether they are misclassified. Thus on the perturbed dataset, the error, which is the fraction of misclassified and not rejected samples, is a more proper notion of such system's performance. For a standard robust classifier, its perturbed set error is computed as the complement of its accuracy on the perturbed set. Using different p-norm and maximum perturbation constrains we trained four detection systems (each has 10 base detectors), with training and validation adversarial examples optimized using PGD attacks of different steps and step-size (see Table 6). At each step of PGD attack we use the Adam optimizer to perform gradient descent, both for L 2 and L ∞ constrained scenarios. Appendix B.1 provides more training details. Robustness . The robustness test in Table 1 confirm that the base detectors trained with objective 5 are able to withstand much stronger PGD attacks, for both L 2 and L ∞ scenarios. Normalized steepest descent is another popular choice for performing PGD attack (; MadryLab, b; a), for which we got similar robustness (Table 8). Further on the complete list of performances of the L ∞ = 0.3 and L ∞ = 0.5 trained detectors, cross-norm and cross-perturbation test , and random restart test are included in Appendix D.1. Table 1: AUC scores of the first two detectors (k = 0, 1) tested with different strengths of PGD attacks using Adam optimizer. PGD attack steps, step-size Detection . Figure 2a shows that the combined attack is the most effective attack against integrated detection. Generative detection (attacked using loss 9) outperforms integrated detection, especially when the detection threshold is low (the region where TPR is high). In Figure 12 we confirm that loss 9 is more effective than CW loss and cross-entropy loss for attacking generative detection. Notably, the red curve that overlaps the y-axis shows that integrated detection can perfectly detect adversarial examples crafted by attacking only the classifier (using objective 8). In Table 2 we compare the performances of our generative detection method with the state-of-the-art detection method as identified by Carlini & Wagner (2017a). Our method using L ∞ = 0.5 trained base detectors is able to outperform the state-of-the-art method by a large margin. Appendix C describes the procedure we used to compute the mean L 2 distortion of our method. Classification . In Figure 2b, we compare the robust classification performance of our methods and a state-of-the-art robust classifier. While the performance of the robust classifier is fixed, by using different rejection thresholds, our classification methods provide the option to balance standard accuracy and robust error. The generative classifier outperforms the integrated classifier when the rejection threshold is low (i.e., when the perturbed set error is high). We observe that a stronger attack (= 0.4) breaks the robust classifier, while the generative classifier still exhibits robustness, even though both systems are trained with the same L ∞ = 0.3 constrain. shows perturbed samples by performing targeted attacks against the generative classifier and robust classifier. We observe that perturbed samples produced by attacking the generative classifier have distinguishably visible features of the target class, indicating that the base detectors, from which the generative classifier is built, have successfully learned the class-conditional distributions, and the perturbation has to change the semantics of the underlying sample for a successful attack. In contrast, perturbations introduced by attacking the robust classifier are not interpretable, even though they could cause high logit output of the target classes (see Figure 13 for the logit outputs distribution). Following this path and use a larger perturbation limit, it is straightforward to generate unrecognizable images that cause highly confident predictions of the robust classifier. Perturbed samples (generative classifier) Perturbed samples (robust classifier) Figure 3: Natural samples and corresponding perturbed samples produced by performing a targeted attack against the generative classifier and robust classifier . Targets from top row to bottom row are digit class from 0 to 9. We perform the targeted attack by maximizing the logit output of the targeted class, using L ∞ = 0.4 constrained PGD attack of steps 100 and step-size 0.01. We note that both classifiers use L ∞ = 0.3 for their training constraint. On CIFAR10 we train the base detectors using L ∞ = 8 constrain PGD attack of steps 40 and step size 0.5. Note that the scale of and step-size here is 0-255 (rather than 0-1 as in the case of MNIST). The robust classifier that we will compare with is trained with the same L ∞ = 8 constraint but with a different step-size (see Appendix D.2.2 for a discussion of step-sizes). Appendix B.2 provides the training details. Robustness . Table 3 shows that the base detector models can withstand attacks that are significantly stronger than the training attack. In Appendix D.2.1 we present random restart test , cross-norm and cross-perturbation test , and robustness test for L 2 based models. Detection . Consistent with the MNIST , in Figure 4a combined attack is the most effective method against integrated detection. Similarly, the generative detection outperforms integrated detection when the detection threshold is low (i.e., where TPR is high). In this figure we use loss 9 to attack generative detection, and in Figure 14 we show that it's more effective than attack based on cross-entropy loss and CW loss. In Table 4 our method outperforms the state-of-the-art adversarial detection method. Classification . In Figure 4b, we did not observe a dramatic decrease in the robust classifier's performance when we increase the perturbation limit to = 12. Integrated classification can reach the standard accuracy of a regular classifier, but at the cost of significantly increasing error on the perturbed set. Figure 5 shows some perturbed samples produced by attacking the generative classifier and robust classifier. While these two classifiers have similar errors on the perturbed set, samples produced by attacking the generative classifier have more visible features of the targets, which indicates that the adversary has to change more semantic in order to cause the same error. Figures 6 and 15 demonstrate that hard to recognize images are able to cause high logit outputs of the robust classifier. Such examples highlight a major defect of the defense mechanisms based on ordinary adversary training: they could be easily fooled by unrecognizable inputs (; b;). In contrast, samples that cause high logit outputs of the generative classifier all have clear semantic meaning. Since both classifiers are trained with L ∞ = 8 constrain, these indicate that AAT improves robust and interpretable feature learning. (See Appendix E for Gaussian noise attack and a discussion about the interpretability of our approach.) The visual similarity between generated samples in Figure 6 and real samples further suggests that detectors have successfully learned the conditional data distributions. Similarly, on ImageNet, we show asymmetrical adversarial training induces detection robustness and supports the learning of class-conditional distributions. Our experiment is based on Restricted ImageNet , a subset of ImageNet that has its samples reorganized into customized categories. The dog category contains images of different dog breeds collected from ImageNet class between 151 and 268. We trained a dog class detector by finetuning a pre-trained ResNet50 model. The dog category covers a range of ImageNet classes, with each one having its logit output. We use the subnetwork defined by the logit output of class 151 as the detector (in principle logit output of other classes in the range should also work). Due to computational constraints, we only validated the robustness of a L ∞ = 0.02 trained detector (trained with PGD attack of steps 40 and step-size 0.001), and we present the in Table 5 Generated by attacking generative classifier Generated by attacking robust classifier Figure 6: Images generated from class-conditional Gaussian noise by performing targeted attack against the generative classifier and robust classier. we use PGD attack of steps 60 and step-size 0.5 × 255 to perform L 2 = 30 × 255 constrained attack (same as . The Gaussian noise inputs from which these two plots are generated are the same. Samples not selected. In this paper, we studied the problem of adversarial detection under the robust optimization framework and proposed a novel adversarial detection scheme based on input space partitioning. Our formulation leads to a new generative modeling technique which we called asymmetrical adversarial training (AAT). AAT's capability to learn class-conditional distributions further gives rise to generative detection/classification methods that show competitive performance and improved interpretability. In particular, our generative classifier is more resistant to "rubbish examples", a significant threat to even the most successful defense mechanisms. High computational cost (see Appendix F for more discussion) is a major drawback of our methods, and in the future, we will explore the idea of shared computation between detectors. A A DEEPER LOOK AT ASYMMETRICAL ADVERSARIAL TRAINING Under the energy-based learning framework , the AAT objective could be understood as learning an energy function that has low energy outputs on target class data points, and high energy outputs anywhere else. The energy function in this case could be defined using the logit output of the target detector:. Using the Gibbs distribution we could obtain a density function and it should be interpreted as the joint distribution of the data point and the corresponding class category p(x, k) =, where Z Θ is a normalizing constant known as the partition function, and could be computed in our case as Z Θ = k exp(−E θ k (x))dx. (We note a similar formulation is used by .) When x is in high dimensional space, Z Θ is intractable, but for generative classification, only knowing the unnormalized probability will suffice. While ordinary discriminative training only learns a good discriminative boundary, AAT is able to learns the underlying density function. In Figure 7 we provide a schematic illustration of this phenomenon ( on real 1D data is in Figure 8). In Figure 9 we observe similar on 2D benchmarking datasets (; ; ; ;) that detectors are able to cover all models of the distributions. We note that unlike GANs that at the global optimum the discriminator converge to uniform solution and thus could not retain density information (a;), by properly constraining the adversary (especially on the perturbation limit), AAT is able to stably push the detector to a good approximation of the underlying density function. Being able to do reliable (implicit) density estimation justifies our energy-based generative classification formulation (we leave the theoretical analysis of the objective's convergence property to future work). The compelling property of AAT generalizes to the case of high dimensional data. In the following, we present CIFAR10 and ImageNet image generation for our models and state-of-the-art GANs models (Figure 10 vs. Figure 11 for CIFAR10, and Figure 21 vs. Figure 23 for ImageNet). While images generated by our models are not necessarily more "realistic", they clearly have captured the structures of the target objects. In particular, the in Figure 21 show that our model is only modeling the main structures of target objects, but not other irrelevant elements like s and other objects. We note that apart from being able to produce state-of-the-art image generation , our approach at the same time provides classification/detection performance guarantee against norm constrained adversarial examples, while other generative models cannot . The ing energy function after training. The blue data distribution has two modes, but the energy function is not able to capture this structure due to its minute effects on the discriminative objective. (c) By solving the inner maximization in objective 5 red points are pushed towards to the low energy region, but (crucially) they are not collapsed to the lowest energy point due to the perturbation limit. The gap between the two sets of blue points is now filled with red points; minimizing the loss ] causes the energy function to be pulled up on this region. (d) The energy function after training is able to capture the internal structure of the blue data. The positive class data (blue points) are sampled from a mixture of Gaussians (mean 0.4 with std 0.01, and mean 0.6 with std 0.005, each with 250 samples). Both the blue and red data has 500 samples. The estimated density function is computed using Gibbs distribution and network logit outputs. PGD attack steps 20, step size 0.05, and perturbation limit = 0.3. Figure 9: 2D datasets (top row, blue points are class 1 data, and red points are class 0 data, both have 1000 data points) and sigmoid outputs of AAT trained models (bottom row). The architecture of the MLP model for solving these tasks is 2-500-500-500-500-500-1. PGD attack steps 10, step-size 0.05, and perturbation limit L ∞ = 0.5. We use 50K samples from the original training set for training and the rest 10K samples for validation, and report test performances based on the epoch-saved checkpoint that gives the best validation performance. All base detectors are trained using a network consisting of two max-pooled convolutional layers each with 32 and 64 filters, and a fully connected layer of size 1024, same as the one used in. At each iteration we sample a batch of 320 samples, from which in-class samples are used as positive samples, and out-of-class samples are used as the source for adversarial examples that will be used as negative samples. To balance positive and negative examples at each batch, we resample the out-of-class set to have same number of samples as in-class set. All base detectors are trained for 100 epochs. Table 6: MNIST dataset PGD attack steps and step-sizes for base detector training and validation. We train our CIFAR10 base detectors using the ResNet50 model . To speedup training, we take advantage of a natural trained classifier: the subnetwork of f that defines the output logit z(f (·)) k is essentially a "detector", that would output high values for samples of class k, and low values for others. Our detector is then trained by finetuning the subnetwork using objective 5. Our pretrained classifier has a test accuracy of 95.01% (fetched from the CIFAR10 adversarial challenge (MadryLab, a)). At each iteration of training we sample a batch of 300 samples, from which in-class samples are used as positive samples, while an equal number of out-of-class samples are used as sources for adversarial examples. Adversarial examples for training L 2 and L ∞ models are both optimized using PGD attack with normalized steepest descent (MadryLab, b). We report based on the best performances on the CIFAR10 test set (thus don't claim generalization performance of the proposed method). We first find the detection threshold T with which the detection system has 0.95 TPR. We construct a new loss function by adding a weighted loss term that measures perturbation size to objective 9 We then use unconstrained PGD attack to optimize L(x). We use binary search to find the optimal c, where in each bsearch attempt if x is a false positive (max i z(H(x)) i = y and max i =y z(H(x)) i > T ) we consider the current c as effective and continue with a larger c. The configurations for performing binary search and PGD attack are detailed in Table 7. The c upper bound is established such that with this upper bound, no samples except those that are inherently misclassified by the generative classifier, could be perturbed as a false positive. With these settings, our MNIST generative detector with L ∞ = 0.3 base detectors reached 0.9962 FPR, generative detector with L ∞ = 0.5 base detectors reached 0.9936 FPR, and CIFAR10 generative detector reached 0.9995 FPR. We note that it's very difficult to find the optimal c in loss 15 using binary search, hence performance based on mean L 2 distortion is not precise, and we encourage future work to measure detection performances based on norm constrained attacks (as in Figure 2a). Table 8: AUC scores of the first two base detectors under different strengths of PGD attacks using normalized steepest descent. The gradient descent rules for L 2 and L ∞ constrained attacks are respectively x n+1 = x n − γ ∇f (xn) ∇f (xn) 2 and x n+1 = x n − γ · sign(∇f (x n)). Table 9: AUC scores of the first two base detectors under cross-norm and cross-perturbation attacks. L ∞ based attacks use steps 200 and step-size 0.01, and L 2 based attacks uses steps 200 and step-size 0.1. Table 10: AUC scores of the first MNIST base detector under fixed start and multiple random restarts attacks. These two tests use the same attack configuration: the L ∞ = 0.5 trained base detector is attacked using L ∞ = 0.5 constrained PGD attack of steps 200 and step-size 0.01, and the L 2 = 5.0 trained base detector is attacked using L 2 = 5.0 constrained PGD attack of steps 200 and step-size 0.1. Table 11: AUC scores of all L ∞ = 0.3 trained base detectors. Tested with L ∞ = 0.3 constrained PGD attacks of steps 200 and step-size 0.01. Table 12: AUC scores of all L ∞ = 0.5 trained base detectors. Tested with L ∞ = 0.5 constrained PGD attacks of steps 200 and step-size 0.01. Table 13: AUC scores of the first CIFAR10 base detector under fixed start and multiple random restarts attacks. The L ∞ = 2.0 base detector is attacked using PGD attack of steps 10 and stepsize 0.5, and the L ∞ = 8.0 base detector is attacked using PGD attack of steps 40 and step-size 0.5. Table 16: AUC scores of L ∞ = 2.0 trained base detectors under L ∞ = 2.0 constrained PGD attack of steps 10 and step-size 0.5. Table 17: AUC scores of L ∞ = 8.0 trained base detectors under L ∞ = 8.0 constrained PGD attack of steps 40 and step-size 0.5. We found training with adversarial examples optimized with a sufficiently small step-size to be essential for detection robustness. In table 18 we tested two L ∞ = 2.0 base detectors respectively trained with 0.5 and 1.0 step-size. The step-size 1.0 model is not robust when tested with a much smaller step-size. We observe that when training the step-size 1.0 model, training set adv AUC reached 1.0 in less than one hundred iterations, but test set natural AUC plummeted to around 0.95 and couldn't recover thereafter. (Please refer to Figure 16 for the definitions of adv AUC and nat AUC.) This suggests that naturally occurring data samples, and adversarial examples produced using a large step-size, live in two quite different data spaces -training a classifier to separate these two kinds of data is easy, but the performance won't generalize to real attacks. was able to train their CIFAR10 robust classifier using step-size 2, we found this step-size not working in our case. To study the effects of perturbation limit on asymmetrical adversarial training, we compare one L ∞ = 2.0 trained and one L ∞ = 8.0 trained base detector. In Figure 16 we show the training and testing history of these two models. The = 2.0 model history show that by adversarial finetuning the model reach robustness in just a few thousands of iterations, and the performance on natural samples is preserved (test natural AUC begins at 0.9971, and ends at 0.9981). Adversarial finetuning on the = 8.0 model didn't converge after an extended 20K iterations of training. The gap between train adv AUC and test adv AUC of the = 8.0 model is more pronounced, and we observed a decrease of test natural AUC from 0.9971 to 0.9909. The comparison shows that training with larger perturbation limit is more time and resource consuming, and could lead to performance decrease on natural samples. The benefit is that the model learns more interpretable features. In Figure 17, perturbations generated by attacking the naturally trained classifier (corresponds to 0 perturbation limit) don't have clear semantic. In contrast, perturbed samples of the L ∞ = 8 model are completely recognizable. Figure 18: ImageNet 224×224×3 random samples generated from class-conditional Gaussian noise by attacking robust classifier and detector models trained with different constrains. Note than large perturbation models are still under training and haven't reached robustness. Please refer to for the detail about how the class-conditional Gaussian is estimated. In this section we use Gaussian noise attack experiment to motivate a discussion about the interpretabilities of our generative classification model and discriminative robust classification model . Our approach is more interpretable in the sense that it provides a probabilitic view of the decision making process of the classification problem, and this probabilistic interpolation is further supported by the experimental . We first discuss how these two approaches determine the posterior class probabilities. For the discriminative classifier, the posterior probabilities are computed from the logit outputs of the classifier using the softmax function p(k|x) = exp(z(f (x)) k ) K j=1 exp(z(f (x))j ). For our generative classifier, the posterior probabilities are computed in two steps: in the first, we train our base detectors, which is the process of solving the inference problem of determining the joint probability p(x, k) (see Appendix A), and in the second, we use Bayes rule to compute the posterior probability p(k|x) =. Coincidentally, the formulas for computing the posterior probabilities take the same form. But in our approach, the exponential of the logit output of a detector (i.e., exp(z(h k (x)))) has a clear probabilistic interpretation: it's the unnormalized joint probability of the input and the corresponding class category. We use Gaussian noise attack to demonstrate that this probabilistic interpretation is consistent with our visual perception. We start from a Gaussian noise image, and gradually perturb it to cause higher and higher logit outputs. This is implemented by targeted PGD attack against logit outputs of these two classification models. The ing images in Figure 24 show that, for our model, the logit output increase direction is the semantic changing direction; while for the discriminative robust model, the perturbed image computed by increasing logit outputs are not as clearly interpretable. In particular, the perturbed images that cause high logit outputs of the robust classifiers are not recognizable. In this section we provide an analysis of the computational cost of our generative classification approach. In terms of memory requirements, if we assume the softmax classifier (i.e., the discriminative robust classifier) and the detectors use the same architecture (i.e., only defer in the final layer) then the detector-based generative classifier is approximately K times more expensive than the Kclass softmax classifier. This also means that the computational graph of the generative classifier is K times larger than the softmax classifier. Indeed, in the CIFAR10 task, on our Quadro M6000 24GB GPU (TensorFlow 1.13.1), the inference speed of the generative classifier is roughly ten times slower than the softmax classifier. We next benchmark the training speed of these two types of classifiers. The generative classifier has K logit outputs, with each one defined by the logit output of a detector. Same with the softmax classifier, except that the K outputs share the parameters in the convolutional part. Now consider ordinary adversarial training on the softmax classifier and asymmetrical adversarial training on the generative classifier. To train the softmax classifier, we use batches of N samples. For the generative classifier, we train each detector with batches of 2 × M samples (M positive samples and M negative samples). At each iteration, we need to respectively compute N and M × K adversarial examples for these two classifiers. Now we test the speed of the following two scenarios: 1) compute the gradient w.r.t. to N samples on a single computational graph, and 2) compute the gradient w.r.t to M × K samples on K computational graphs, with each graph working on M samples. We assume in scenario 2 that all the computational graphs are loaded to GPUs, and their computations are in parallel. In our CIFAR10 experiment, we used batches consisting of 30 positive samples and 30 negative samples to train each ResNet50 based detectors. , the softmax classifier was trained with batches of 128 samples. In this case, K = 10, M = 30, and N = 128. On our GPU, scenario 1 took 683 ms ± 6.76 ms per loop, while scenario 2 took 1.85 s ± 42.7 ms per loop. In this case, we could expect asymmetrical adversarial training to be about 2.7 times slower than ordinary adversarial training, if not considering parameter gradient computation. (If we choose to use a large batch size the computational cost will increase accordingly.)

A new generative modeling technique based on asymmetrical adversarial training, and its applications to adversarial example detection and robust classification

represent this text to find a much much longer article about it with say hundreds-thousands of words.

This work aims to provide quantitative answers to the relative importance of concepts of interest via concept activation vectors (CAV). In particular, this framework enables non-machine learning experts to express concepts of interest and test hypotheses using examples (e.g., a set of pictures that illustrate  the concept). We show that CAV can be learned given a relatively small set of examples. Hypothesis testing with CAV can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other sets of concepts. Interpreting networks with CAV does not require any retraining or modification of the network.