Question: Visual attention mechanisms have been widely used in image captioning models. In this paper, to better link the image structure with the generated text, we replace the traditional softmax attention mechanism by two alternative sparsity-promoting transformations: sparsemax and Total-Variation Sparse Attention (TVmax). With sparsemax, we obtain sparse attention weights, selecting relevant features. In order to promote sparsity and encourage fusing of the related adjacent spatial locations, we propose TVmax. By selecting relevant groups of features, the TVmax transformation improves interpretability. We present in the Microsoft COCO and Flickr30k datasets, obtaining gains in comparison to softmax. TVmax outperforms the other compared attention mechanisms in terms of human-rated caption quality and attention relevance. The goal of image captioning is to generate a fluent textual caption that describes a given image (; ; ;). Image captioning is a multimodal task: it combines text generation with the detection and identification of objects in the image, along with their relations. While neural encoder-decoder models have achieved impressive performance in many text generation tasks;; ), it is appealing to design image captioning models where structural bias can be injected to improve their adequacy (preservation of the image's information), therefore strengthening the link between their language and vision components. State-of-the-art approaches for image captioning (a; b; ;) are based on encoder-decoders with visual attention. These models pay attention either to the features generated by convolutional neural networks (CNNs) pretrained on image recognition datasets, or to detected bounding boxes. In this paper, we focus on the former category: visual attention over features generated by a CNN. Without explicit object detection, it is up to the attention mechanism to identify relevant image regions, in an unsupervised manner. A key component of attention mechanisms is the transformation that maps scores into probabilities, with softmax being the standard choice. However, softmax is strictly dense, i.e., it devotes some attention probability mass to every region of the image. Not only is this wasteful, it also leads to "lack of focus": for complex images with many objects, this may lead to vague captions with substantial repetitions. Figure 1 presents an example in which this is visible: in the caption generated using softmax (top), the model attends to the whole image at every time step, leading to a repetition of "bowl of fruit." This undesirable behaviour is eliminated by using our alternative solutions: sparsemax (middle) and the newly proposed TVMAX (bottom). In this work, we introduce novel visual attention mechanisms by endowing them with a new capability: that of selecting only the relevant features of the image. To this end, we first propose replacing softmax with sparsemax . While sparsemax has been previously used in NLP for attention mechanisms over words, it has never been applied to computer vision to attend over image regions. With sparsemax, the attention weights obtained are sparse, leading to the selection (non-zero attention) of only a few relevant features. Second, to further encourage the weights of related adjacent spatial locations to be the same (e.g., parts of an object), we introduce a new attention mechanism: Total-Variation Sparse Attention (which we dub TVMAX), inspired by prior work in structured sparsity . With TVMAX, sparsity is allied to the ability of selecting compact regions. According to our human evaluation experiments, Figure 1: Example of captions generated using softmax (top), sparsemax (middle) and TVMAX attention (bottom). Shading denotes the attention weight, with white for zero attention. The darker the green is, the higher the attention weight is. The full sequences are presented in Appendix C. this leads to better interpretability, since the model's behaviour is better understood by looking at the selected image regions when a particular word is generated. It also leads to a better selection of the relevant features, and consequently to the improvement of the generated captions. This paper introduces three main contributions: • We propose a novel visual attention mechanism using sparse attention, based on sparsemax , that improves the quality of the generated captions and increases interpretability. • We introduce a new attention mechanism, TVMAX, that encourages sparse attention over contiguous 2D regions, giving the model the capability of selecting compact objects. We show that TVmax can be evaluated by composing a proximal operator with a sparsemax projection, and we provide a closed-form expression for its Jacobian. This leads to an efficient implementation of its forward and backward pass. • We perform an empirical and qualitative comparison of the various attention mechanisms considered. We also carry out a human evaluation experiment, taking into account the generated captions as well as the perceived relevance of the selected regions. Attention mechanisms have the ability to select the relevant features, in this case spatial locations. This requires a mapping from importance scores to a distribution,, p 0 denotes the simplex (the set of all probability distributions over k values). The standard choice for this mapping is softmax, defined as: However, as softmax is strictly positive, its output is dense. Thus, the model must pay some attention to the whole image and, consequently, assign lower attention weights to the relevant regions. This motivates our proposed selective visual attention mechanisms, which, by being sparse, are able to better isolate the relevant image regions. To achieve selective capabilities, we propose the use of sparsemax , a sparse mapping consisting in the Euclidean projection of z onto the probability simplex: which allows to obtain sparse outputs with a small increase in complexity. Output sparsity is an attractive property for attention mechanisms, since some features do not provide relevant information for the current prediction. In the image captioning case, using sparsemax allows focusing only on the spatial locations of the image that are relevant to the word being generated, assigning zero attention weight to all other regions. To generate descriptive captions, the model should identify the objects present in the image. Thus, when generating object-related words, the attention mechanism should assign high weights to the regions of the image containing the object. However, sparsemax is unstructured and index-invariant, leading it to select discontinuous regions. To overcome this, we propose a new visual attention mechanism, TVMAX. TVMAX is a non-trivial generalization of fusedmax , a transformation based on fused lasso, to the 2D case. To this end, we first extend fusedmax even more generally, to arbitrary graphs. Let w ∈ R k, and let I = {1, . . ., k}. Consider a graph over I defined by its edges E ⊆ I × I, where an edge between i and j means we want to encourage w i to be close to w j. For simplicity we use i ∼ j as shorthand for (i, j) ∈ E. The generalized fused lasso penalty is defined as: Minimizing Ω E encourages "fused" solutions, i.e., it encourages w i = w j for i ∼ j. In particular, its proximal operator 1 can be seen as a fused signal approximator, seeking a vector w that approximates z well (in terms of Euclidean distance) and that is encouraged to be fused: Computing the value of prox λΩ E is non-trivial in general , but for certain edge configurations, described below, efficient algorithms exist. • If E forms a chain, i.e. i ∼ j ⇐⇒ i = j − 1, the problem is called 1D total variation and can be solved in O(k) time using the taut string algorithm . We use the quasilinear algorithm of , which is very fast in practice. • If the indices are aligned on a 2D grid, as in an image, and i ∼ j holds iff. j is to the right or immediately below i, the problem is called 2D total variation. Unlike the 1D case, exact algorithms are not available. However, for an input of size a × b, it is possible to split the penalty into a column-wise and b row-wise 1D problems. We may then apply a number of iterative methods, for instance proximal Dykstra . TVMAX combines 2D total variation (TV2D) regularization with sparsemax. This way it promotes sparsity and encourages the attention weights of adjacent spatial locations to be the same, selecting contiguous regions of the image. TVMAX is defined as follows: Definition 1 (TVMAX). Let z ∈ R k, such that z's indices can be decomposed into rows and columns. The TVMAX transformation is defined as where λ is an hyper-parameter controlling the amount of fusion (λ = 0 recovers sparsemax) and Ω T V 2D is a 2D total variation penalty. Note that Eq. 5 differs from Eq. 4 in which the variable p is further constrained to lie in the probability simplex. We show next how the forward and backward passes can be efficiently computed. To construct generalized fused sparse attention, we follow and define This can be seen as a constrained fused lasso approximator, because the solution p must be a probability distribution vector. While the optimization function is very similar to Eq. 4, the additional constraint that p ∈ increases complexity. Fortunately, the following holds: Proposition 1 (Computing generalized fusedmax). The proof is given in Appendix A.2. Proposition 1 also provides a shortcut for deriving the Jacobian of generalized fusedmax via the chain rule: denoting by J F the Jacobian of prox λΩ E, we have As we already know how to compute J sparsemax (Appendix A.1), we may concentrate our effort on deriving the simpler J F (Eq. 9). Proposition 2 (Group-wise characterization of prox λΩ E). Let w:= prox λΩ E, and denote by G i the set of indices fused to w i in the solution, G i may be defined recursively: Define s ij = sign(w i − w j). Then, the solution has the expression Proposition 2 shows how to easily compute a generalized Jacobian of gfusedmax: since small perturbations in z never change the groups G i nor the signs of across-group differences s ij, differentiating Eq. 8 yields This generalizes Lemma 1 of to generalized fused lasso, with a simpler proof, given in Appendix A.3. As we show in Proposition 1, computing TVMAX's forward pass can be done by chaining efficient algorithms for TV2D and sparsemax. From Eq. 7 we have that TVMAX's Jacobian can be computed as J TVMAX = J sp (prox λΩ T V (z))J tv (z), where J sp is the sparsemax's Jacobian and J tv is the Jacobian of the Total Variation proximal operator. 3 As derived in Proposition 2, (J tv) i,j = 1 /nij if i and j are fused in a group with n ij elements, and 0 otherwise. The backward pass intuitively involves "spreading" the credit assigned to one image location evenly across all locations fused with it. This can be implemented by Algorithm 1 in O(k+N g log k) where N g is the number of groups of fused positions. In the worst case, when there are no positions fused, the complexity is O(k + k log k). This algorithm is inspired by flood filling algorithms . Algorithm 1 TVMAX backward pass (Jacobian-vector products) 1 Input: p = TVMAX(z), dp ∈ R k. 2 Output: dz = J TVMAX (dp) dp # Eqs. 14 and 15 of §A.1 8 while |V | < k do # check if all positions have been visited if (i, j) ∈ V then push (i, j) to N # add not visited neighbours of (i, j) to the stack 17 if G not empty then: To compare the proposed attention mechanisms, we use a straight-forward simple encoder-decoder model with visual attention, inspired by Liu et al. (2018a). The model is sketched in Figure 2. Given an image, we use a residual CNN pretrained on ImageNet to get a feature map with spatial dimension of size 8 × 8 and channel dimension of size 2048, that go through a fine-tuned feedforward layer yielding g = 512 feature maps. The visual feature matrix V = [v 1, v 2, . . ., v k], with v i ∈ R g and k = 64 = 8 × 8, contains the image information used to generate the corresponding caption. Following Liu et al. (2018a), we use input and output attention to select the relevant features for the current generation. To generate the word at position t, the input attention, α t, is computed using the LSTM's previous hidden state, h t−1 ∈ R d. First, a similarity score z t,i, i ∈ {1, . . ., k}, is computed between h t−1 and the i th image cell via a feedforward transformation, as z t,i = w tanh(affine([v i ; h t−1])), for all k image cells. Then, α t is obtained by normalizing the k-dimensional vector of scores z t with softmax, α t = softmax(z t). Using these attention weights, a vector representation of the image to be used as input of the LSTM, is obtained, s t = V α t. The output attention α t, is computed in the same way as above, but applied to the current LSTM hidden state h t, instead of h t−1, and normalized with the different proposed transformations. This produces output visual features s t = V α t, which are passed through a feedforward layer to yield the image representation r t = tanh(affine( s t)). Finally, the predictive probability of the next word is: Settings. The input images are resized to 256 × 256 before going through the residual CNN and the feature maps obtained have a size of 8 × 8. We use an LSTM hidden size of d = 512 and a word embedding size of 256, for all models. The models were trained for 50 epochs using the Adam optimizer with a learning rate of 0.0001 and a decay of 0.8 and 0.999 for the first and second momentum, respectively. After the 10 th epoch, the learning rate starts decaying with a decay factor of 0.99. For TVMAX, we set λ = 0.01. Datasets and Metrics. We report our on the Microsoft COCO (MSCOCO) and Flickr30k datasets. MSCOCO is composed of 113,287 images of common objects in context while Flickr30k consists in 31,000 pictures of people involved in everyday activities and events. Each image is annotated with 5 captions. We use the split proposed by , which stipulates equal validation and test sizes of 5,000 images (MSCOCO) and 1,000 (Flickr30k). The metrics we report are SPICE , CIDEr , longest common subsequence ROUGE, (denoted ROUGE L ;), 1-to 4-gram BLEU (denoted BLEU 4 ;), and METEOR . To investigate whether selective attention alleviates repetition, we also measure the n-gram repetition metric REP . Automated metrics. As can be seen in table 1, overall sparsemax and TVMAX attention mechanisms achieve better when compared with softmax, indicating that the use of selective attention leads to better captions. This improvement does not come at a high computational cost: at Figure 3: Example of captions generated using softmax (top), sparsemax (middle) and TVMAX attention (bottom). Shading denotes the attention weight, with white for zero attention. The darker the green is, the higher the attention weight is. The full sequences are presented in Appendix C. inference time, models using TVMAX and sparsemax are only 1.3x and 1.1x slower than softmax. Moreover, for TVMAX, automatic metrics are slightly worse than sparsemax but still superior to softmax on MSCOCO and similar on Flickr30k. We show next that this is compensated with fewer repetitions and higher scores in the human evaluation of the captions and attention relevance. Human rating. The caption evaluation consisted in attributing a score from 1 to 5 to the caption of each model while the attention evaluation consisted in scoring the relevancy of the attended areas, from 1 to 5, when generating the non stop words of the captions. A full description of the human assessment can be found in Appendix B. Despite performing slightly worse than sparsemax under automated metrics, TVMAX outperforms sparsemax and softmax in the caption human evaluation and the attention relevance human evaluation, reported in Table 2. The superior score on attention relevance shows that TVMAX is better at selecting the relevant features and its output is more interpretable. Additionally, the better caption evaluation demonstrate that the ability to select compact regions induces the generation of better captions. We next explore possible explanations for the TVMAX superior . Repetition. Figure 1 illustrates that softmax attention is prone to spuriously repeating references to the same object. Selective attention mechanisms like sparsemax and especially TVMAX reduce repetition, as measured by the REP metric reported in Table 1. This expected success can be attributed to the sparsity of the attention weights distribution and to the ability to select compact regions exclusively and can be one of the causes of the human evaluation . This happens even though TVMAX generates longer sentences than sparsemax and softmax (9.5 against 9.0 words on average) and shows the benefit of promoting structured and sparse attention simultaneously. To corroborate our intuition that sparsity leads to less repetition, we measured the Jensen-Shannon divergence (JS) between the attention distributions for each step of the generation of the captions correspondent to the images of the MSCOCO test set. The mean JS values are 0.12, 0.29, and 0.34 for softmax, sparsemax, and TVmax, respectively. This shows that sparsity leads to less similar attention distributions along the generation of the captions and, consequently, to less repetitions. Object detection. Using the MSCOCO object detection ground truth, we compared the percentage of objects present in the image that are referred to in the captions, using each attention mechanism. With TVMAX 28.2% of the reference objects are referred, against 27.5% and 27.4% for sparsemax and softmax, repectively. This shows that promoting high attention to groups of spatial locations of the image leads to a more precise identification of the objects. Sparsity. The average image area that receives zero attention is 34% for sparsemax and 25% for TVMAX. To illustrate where the models attend to, we display the output attention in Figures 1 and 3. As expected, softmax weights are spread widely across the image, ending up missing the relevant regions. In contrast, sparsemax and TVMAX weights are zero for the non-relevant spatial locations. Qualitative comparison. As the image of Figure 1 contains various similar objects, the softmax model (top) generates a incoherent, repetition-laden caption. In contrast, the sparsemax (middle) and TVMAX (bottom) models better identify the relevant parts of the image, generating coherent and descriptive captions. Moreover, the groups obtained with TVMAX are clearly visible and more aligned to object boundaries, offering better interpretability, as revealed by human attention assessment. In Figure 3 it can also be noticed that with TVMAX (bottom) the model correctly identified "a group of people" instead of "a soccer player" as with sparsemax (middle) and softmax (top). This indicates its superior ability to correctly define the relevant groups of features and that this ability leads to improved captions. Image captioning. In the last years, neural models with visual attention mechanisms have been receiving increased interest. Several researchers have been studying diverse attention mechanisms in order to refine visual information for image captioning. proposed the use of hard attention, which only attends to one region at each step. However, to generate descriptive captions the model should, often, focus on more than one region. In addition, hard attention is non-differentiable, requiring imitation learning or Monte Carlo policy gradient approximations. proposed bottom-up attention, using an object detection model designed to identify bounding boxes of objects, and top-down attention, selecting the relevant bounding-boxes. proposed an hierarchical attention network composed by a patch detector, object detector, and concept detector. Using object detection models is less demanding on the attention mechanism, since it only has to select the boxes the model should attend to. However, such models are limited by the bounding boxes position's accuracy. introduced a deliberate attention network to refine the attended visual features. Yet, the attention distribution remained dense. Sparse attention. In several tasks only a few features are relevant for the current prediction. This can be attained when using sparse attention. Various prior works have proposed sparse attention mechanisms with promising , (; ; ;). proposed 1D fusedmax, which incorporates the fused lasso, so that adjacent words are encouraged to have the same attention weight. In this work, the authors were able to improve interpretability without sacrificing performance, obtaining superior on textual entailment and summarization. We derive a generalized fused attention mechanism, extending 1D fusedmax. We propose using sparse and structured visual attention, in order to improve the process of selecting the features relevant to the caption generation. For that, we used sparsemax and introduced TVMAX. Results on the image captioning task, show that the attention mechanism is able to select better features when using sparsemax or TVMAX. Furthermore, in the human assessment and attention analysis we see that the improved selection of the relevant features as well as the ability to group spatial features lead to the generation of better captions, while improving the model's interpretability. In future work, TVMAX attention can be applied to other multimodal problems such as visual question answering. It can also be applied in other tasks for which we have prior knowledge of the data's stucture, for instance graphs or trees. Summing up the Eq. 17 over all j ∈ G, we observe that for any k ∈ G, the term λt jk appears twice with opposite signs. Thus, Dividing by |G| gives exactly Eq. 8. This reasoning applies to any group G i. To perform the human evaluation firstly 100 images were randomly selected from the test set of the MSCOCO dataset (using the split proposed by). For each of the selected images, the human evaluators selected a score from 1 to 5 for the captions generated by the models using softmax attention, sparsemax attention, and TVMAX attention. They were also asked to evaluate whether the models attend to the relevant regions of the image when generating a certain word. For that they observed the attention plots corresponding to the non stop words of the caption of each of the models. While in Figures 1 and 3, 4, and 5 we emphasized sparsity with a hard white mask, for the human evaluation the sparse regions of the attention plots were simply fully transparent, to avoid biasing the evaluators. The possible scores were also between 1 and 5. The 100 images were judged by 6 persons both for the captions evaluation and attention evaluation. The order of the captions and attention plots was randomly chosen for each image. With these scores, we computed the mean of the captions evaluation scores and the mean of the attention relevance evaluation scores. The are reported in Table 2. C ADDITIONAL ATTENTION VISUALIZATION Figure 4: Example generated captions using softmax attention (top), sparsemax attention (middle) and TVMAX attention (bottom). The captions are "A bowl of fruit and a bowl of fruit", "A bowl of fruit and oranges on a table" and "A bowl of oranges and a banana on a table". Figure 5: Example generated captions using softmax attention (top), sparsemax attention (middle) and TVMAX attention (bottom). The captions are "A soccer player is running to the base", "A soccer player is running to the field" and "A group of people playing soccer on a field".

Represent the TLDR for finding the corresponding scientific article!

We propose a new sparse and structured attention mechanism, TVmax, which promotes sparsity and encourages the weight of related adjacent locations to be the same.

This paper 1) characterizes functions representable by ReLU DNNs, 2) formally studies the benefit of depth in such architectures,  3) gives an algorithm to implement empirical risk minimization to global optimality for two layer ReLU nets.


Question: Represent the text to find a very short summary (1-2 sentences max)

Question: Although there are more than 65,000 languages in the world, the pronunciations of many phonemes sound similar across the languages. When people learn a foreign language, their pronunciation often reflect their native language's characteristics. That motivates us to investigate how the speech synthesis network learns the pronunciation when multi-lingual dataset is given. In this study, we train the speech synthesis network bilingually in English and Korean, and analyze how the network learns the relations of phoneme pronunciation between the languages. Our experimental shows that the learned phoneme embedding vectors are located closer if their pronunciations are similar across the languages. Based on the , we also show that it is possible to train networks that synthesize English speaker's Korean speech and vice versa. In another experiment, we train the network with limited amount of English dataset and large Korean dataset, and analyze the required amount of dataset to train a resource-poor language with the help of resource-rich languages. every speaker in the trained model could speak both English and Korean fluently. We investigated 37 whether the phoneme embeddings from different languages are learned meaningful representations. We found phonemes with similar pronunciation tend to stay closer than the others even across the 39 different languages. From these , we thought that the cross-lingual model would be possible to 40 generalize for a language with scarce amount of data when there is another language with abundant 41 data. We trained cross-lingual TTS models while differing the amount of data for a resource-scarce 42 language. Then we computed and compared character error rate (CER) of generated speeches from 43 each model by crowd-sourced human dictation. To summarize, the contributions of this study are as follows: 45 1. We successfully trained a cross-lingual multi-speaker TTS model using English and Korean 46 data in which no bilingual speaker is included. 3. We showed how much data of a language is required to train a TTS model when we have a 50 large amount of data from another language. DISPLAYFORM0 The 4 Spectrogram of generated speech using English phonemes and the nearest Korean phonemes using crowd-sourcing platform, and the average CER of the transcriptions is reported in TAB2. Also, the generated samples from each model are posted in the demo page. The symbols in the parentheses are IPA symbols. The symbol'-' in the parentheses denotes that there was no IPA symbol for that phoneme.184 TAB2 aa (a) AE2 (ae) AY0 (aI) AW0 (aU) AW1 (aU) DISPLAYFORM0 The symbols in the parentheses are IPA symbols. DISPLAYFORM1

Represent the short summary for finding the corresponding scientific article (long)

Learned phoneme embeddings of multilingual neural speech synthesis network could represent relations of phoneme pronunciation between the languages.

This work proposed a universal visual representation for neural machine translation (NMT) using retrieved images with similar topics to source sentence,  extending image applicability in NMT.


Question:
represent the scientific article to find a short tldr

Given Low bit-width weights and activations are an effective way of combating the increasing need for both memory and compute power of Deep Neural Networks. In this work, we present a probabilistic training method for Neural Network with both binary weights and activations, called PBNet. By embracing stochasticity during training, we circumvent the need to approximate the gradient of functions for which the derivative is zero almost always, such as $\textrm{sign}(\cdot)$, while still obtaining a fully Binary Neural Network at test time. Moreover, it allows for anytime ensemble predictions for improved performance and uncertainty estimates by sampling from the weight distribution. Since all operations in a layer of the PBNet operate on random variables, we introduce stochastic versions of Batch Normalization and max pooling, which transfer well to a deterministic network at test time. We evaluate two related training methods for the PBNet: one in which activation distributions are propagated throughout the network, and one in which binary activations are sampled in each layer. Our experiments indicate that sampling the binary activations is an important element for stochastic training of binary Neural Networks. Deep Neural Networks are notorious for having vast memory and computation requirements, both during training and test/prediction time. As such, Deep Neural Networks may be unfeasible in various environments such as battery powered devices, embedded devices (because of memory requirement), on body devices (due to heat dissipation), or environments in which constrains may be imposed by a limited economical budget. Hence, there is a clear need for Neural Networks that can operate in these resource limited environments. One method for reducing the memory and computational requirements for Neural Networks is to reduce the bit-width of the parameters and activations of the Neural Network. This can be achieved either during training (e.g., BID15 ; BID0) or using post-training mechanisms (e.g., BID15, BID5). By taking the reduction of the bit-width for weights and activations to the extreme, i.e., a single bit, one obtains a Binary Neural Network. Binary Neural Networks have several advantageous properties, i.e., a 32× reduction in memory requirements and the forward pass can be implemented using XNOR operations and bit-counting, which in a 58× speedup on CPU BID20. Moreover, Binary Neural Networks are more robust to adversarial examples BID2. BID21 introduced a probabilistic training method for Neural Networks with binary weights, but allow for full precision activations. In this paper, we propose a probabilistic training method for Neural Networks with both binary weights and binary activations, which are even more memory and computation efficient. In short, obtain a closed form forward pass for probabilistic neural networks if we constrain the input and weights to binary (random) variables. The output of the Multiply and Accumulate (MAC) operations, or pre-activation, is approximated using a factorized Normal distribution. Subsequently, we introduce stochastic versions of Max-Pooling and Batch Normalization that allow us to propagate the pre-activatoins throughout a single layer. By applying the sign(·) activation function to the random pre-activation, we not only obtain a distribution over binary activations, it also allows for backpropagation through the sign(·) operation. This is especially convenient as this in a deterministic Neural Network all gradient information is zeroed out when using sign as activation. We explore two different methods for training this probabilistic binary neural network: In the first method the activation distribution of layer l is propagated to layer (l + 1), which means the MAC operation is performed on two binary random variables. In the second method the binary activation is sampled as the last operation in a layer using the concrete relaxation BID16. This can be thought of as a form of local reparametrization BID11. We call the networks obtained using these methods PBNet and PBNet-S, respectively. At test time, we obtain a single deterministic Binary Neural Network, an ensemble of Binary Neural Networks by sampling from the parameter distribution, or a Ternary Neural Network based on the Binary weight distribution. An advantage of our method is that we can take samples from the parameter distribution indefinitely-without retraining. Hence, this method allows for anytime ensemble predictions and uncertainty estimates. Note that while in this work we only consider the binary case, our method supports any discrete distribution over weights and activations. Algorithm 1: Pseudo code for forward pass of single layer in PBNet(-S). a l−1 denotes the activation of the previous layer, B the random binary weight matrix, τ is the temperature used for the concrete distribution, f (·, ·) the linear transformation used in the layer, > 0 a small constant for numerical stability, D the dimensionality of the inner product in f, and γ & β are the parameters for batch normalization. DISPLAYFORM0 // Max pooling if max pooling required then n ∼ N (0, I); s = µ + σ n; ι = max-pooling-indices(s); µ, σ 2 = select-at-indices(µ, σ 2, ι); end // Binarization and sampling DISPLAYFORM1 In this section we introduce the probabilistic setting of the PBNet. Moreover, the approximation of the distribution on the pre-activations is introduced. For an explanation of the other operations in the PBNet, see Section 2.1 for the activation, Section 2.1.1 for the sampling of activations, and Section 2.2 for Pooling and Normalization. We aim to train a probabilistic Binary Neural Network. As such, we pose a binary distribution over the weights of the network and optimize the parameters of this distribution instead of the parameters directly. This way, we obtain a distribution over parameters, but also deal with the inherent discreteness of a Binary Neural Network. Given an objective function L(·), this approach can be thought of in terms of the variational optimization framework BID23. Specifically, by optimizing the parameters of the weight distributions, we optimize a bound on the actual loss: min DISPLAYFORM2 where B are the binary weights of the network and q θ (B) is a distribution over the binary weights. For q θ (B) a slight reparametrization of the Bernoulli distribution is used, which we will refer to as the Binary distribution. This distribution is parameterized by θ ∈ [−1, 1] and is defined by: DISPLAYFORM3 For the properties of this distribution, please refer to Appendix A.We will now consider using the Binary distribution for both the weights and the activations in a Neural Network. Since the pre-activations in a Neural Network are computed using MAC operations, which are the same for each value in the pre-activation, we will only consider a single value in our discussion here. Let w ∼ Binary(θ) and h ∼ Binary(φ) be the weight and input random variable for a given layer. As such, the innerproduct between the weights and input is distributed according to a translated and scaled Poisson binomial distribution: DISPLAYFORM4 Where D is the dimensionality of h and w and denotes element-wise multiplication. See the picket fence on the top in FIG0 for an illustration of the PMF of a Poisson binomial distribution. Although the scaled and translated Poisson binomial distribution is the exact solution for the inner product between the weight and activation random variables, it is hard to work with in subsequent layers. For this reason, and the fact that the Poisson binomial distribution is well approximated by a Normal distribution , we use a Normal approximation to the Poisson binomial distribution, which allows for easier manipulations. Using the properties of the Binary distribution and the Poisson binomial distribution, the approximation for the pre-activation a is given by: DISPLAYFORM5 Note that, this is exactly the approximation one would obtain by using the Lyapunov Central Limit Theorem (CLT), which was used by BID21. This allows us to obtain a close approximation to the pre-activation distribution, which we can propagate through the layer and/or network. So far, only the MAC operation in a given layer is discussed, in Section 2.1 application of the binary activation is discussed and in Section 2.1. The stochastic versions of Batch Normalization and Max Pooling are introduced in Section 2.2. For specifics on sampling the binary activation, see Section 2.1.1. The full forward pass for a single layer is given in detail in Algorithms 1. Since the output of a linear operation using binary inputs is not restricted to be binary, it is required to apply a binarization operation to the pre-activation in order to obtain binary activations. Various works -e.g., BID7 and BID20 -use either deterministic or stochastic binarization functions, i.e., DISPLAYFORM0 +1 with probability p = sigmoid(a) −1 with probability 1 − p.In our case the pre-activations are random variables. Hence, applying a deterministic binarization function to a random pre-activations in a stochastic binary activation. Specifically, let a i ∼ N (µ i, σ 2 i) be a random pre-ctivation obtained using the normal approximation, as introduced in the previous section, then the activation (after binarization) is again given as a Binary random variable". Interestingly, the Binary probability can be computed in closed form by evaluating the probability density that lies above the binarization threshold: DISPLAYFORM1 where Φ(·|µ, σ 2) denotes the CDF of N (µ, σ 2). Applying the binarization function to a random pre-activation has two advantages. First, the derivatives ∂q i /∂µ i and ∂q i /∂σ i are not zero almost everywhere, in contrast to the derivatives of b det and b stoch when applied to a deterministic input. Second, the distribution over h i reflects the true uncertainty about the sign of the activation, given the stochastic weights, whereas b stoch uses the magnitude of the pre-activation as a substitute. For example, a pre-activation with a high positive magnitude and high variance will be deterministically mapped to 1 by b stoch. In contrast, our method takes the variance into account and correctly assigns some probability mass to −1. See FIG0 for a graphical depiction of the stochastic binary activation. So far, we have discussed propagating distributions throughout the network. Alternatively, the binary activations can be sampled using the Concrete distribution BID16 during training. specifically, we use the hard sample method as discussed by BID9. By sampling the activations, the input for subsequent layers will match the input that is observed at test time more closely. As a consequence of sampling the activation, the input to a layer is no longer a distribution but a h ∈ {−1, +1} D vector instead. As such, the normal approximation to the pre-activation is computed slightly different. From the Lyapunov CLT it follows that the approximation to the distribution of the pre-activation is given by: DISPLAYFORM0 where w ∼ Binary(θ) is a random weight. Similarly, the pre-activation of the input layer is also computed using this approximation-given a real-valued input vector. We will refer to a PBNet that uses activation sampling as PBNet-S. Other than a linear operation and an (non-linear) activation function, Batch Normalization BID8 and pooling are two popular building blocks for Convolutional Neural Networks. For Binary Neural Networks, applying Batch Normalization to a binarized activation will in a non-binary . Moreover, the application of max pooling on a binary activation will in a feature map containing mostly +1s. Hence, both operations must be applied before binarization. However, in the PBNet, the binarization operation is applied before sampling. As a consequence, the Batch Normalization and pooling operations can only be applied on random pre-activations. For this reason, we define these methods for random variables. Although there are various ways to define these operation in a stochastic fashion, our guiding principle is to only leverage stochasticity during training, i.e., at test time, the stochastic operations are replaced by their conventional implementations and parameters learned in the stochastic setting must be transferred to their deterministic counterparts. Batch Normalization (BN) BID8 -including an affine transformation -is defined as follows: DISPLAYFORM0 where a i denotes the pre-activation before BN,â the pre-activation after BN, and m & v denote the sample mean and variance of DISPLAYFORM1, for an M -dimensional pre-activation. In essence, BN translates and scales the pre-activations such that they have approximately zero mean and unit variance, followed by an affine transformation. Hence, in the stochastic case, our aim is that samples from the pre-activation distribution after BN also have approximately zero mean and unit variance-to ensure that the stochastic batch normalization can be transfered to a deterministic binary neural network. This is achieved by subtracting the population mean from each pre-activation random variable and by dividing by the population variance. However, since a i is a random variable in the PBNet, simply using the population mean and variance equations will in non-standardized output. Instead, to ensure a standardized distribution over activations, we compute the expected population mean and variance under the pre-activation distribution: DISPLAYFORM2 where M is the total number of activations and a i ∼ N (µ i, σ i) are the random pre-activations. By substituting m and v in Equation 8 by Equation 9 and 10, we obtain the following batch normalized Gaussian distributions for the pre-activations: DISPLAYFORM3 Note that this assumes a single channel, but is easily extended to 2d batch norm in a similar fashion as conventional Batch Normalization. At test time, Batch Normalization in a Binary Neural Network can be reduced to an addition and sign flip of the activation, see Appendix B for more details. In general, pooling applies an aggregation operation to a set of (spatially oriented) pre-activations. Here we discuss max pooling for stochastic pre-activations, however, similar considerations apply for other types of aggregation functions. In the case of max-pooling, given a spatial region containing stochastic pre-activations a 1,..., a K, we aim to stochastically select one of the a i. Note that, although the distribution of max(a 1, . . ., a K) is well-defined BID18, its distribution is not Gaussian and thus does not match one of the input distributions. Instead, we sample one of the input random variables in every spatial region according to the probability of that variable being greater than all other variables, i.e., ρ i = p(a i > z \i), where z \i = max({a j} j =i ). ρ i could be obtained by evaluating the CDF of (z \i − a i) at 0, but to our knowledge this has no analytical form. Alternatively, we can use Monte-Carlo integration to obtain ρ: DISPLAYFORM0 where one-hot(i) returns a K-dimensional one-hot vector with the ith elements set to one. The pooling index ι is then sampled from Cat(ρ). However, more efficiently, we can sample s ∼ p(a 1, . . ., a K) and select the index of the maximum in s, which is equivalent sampling from Cat(ρ). Hence, for a given max pooling region, it is sufficient to obtain a single sample from each normal distribution associated with each pre-activation and keep the random variable for which this sample is maximum. A graphical overview of this is given in Figure 2.Other forms of stochastic or probabilistic max pooling were introduced by BID13 , however, in both cases a single activation is sampled based on the magnitude of the activations. In contrast, in our procedure we stochastically propagate one of the input distributions over activations. For the PBNet the parameters θ for q θ (B) are initialized from a uniform U (−1, 1) distribution. Although the final parameter distribution more closely follows a Beta(α, α) distribution, for α < 1, we did not observe any significant impact choosing another initialization method for the PBNet. In the case of the PBNet-S, we observed a significant improvement in training speed and performance by initializing the parameters based on the parameters of a pre-trained full precission Neural Network. This initializes the convolutional filters with more structure than a random initialization. This is desirable as in order to flip the value of a weight, the parameter governing the weight has to pass through a high variance regime, which can slow down convergence considerably. Select maximum per region 2 Sample from input distributions 1Keep maximum distribution for each region 3Figure 2: Max pooling for random variables is performed by taking a single sample from each of the input distributions. The output random variable for each pooling region is the random variable that is associated with the maximum sample. For the PBNet-S, We use the weight transfer method introduced by BID21 in which the parameters of the weight distribution for each layer are initialized such that the expected value of the random weights equals the full precision weight divided by the standard deviation of the weights in the given layer. Since not all rescaled weights lay in the [−1, 1] range, all binary weight parameters are clipped between [−0.9, 0.9]. This transfer method transfers the structure present in the filters of the full precision network and ensures that a significant part of the parameter distributions is initialized with low variance. In our training procedure, a stochastic neural network is trained. However, at test time (or on hardware) we want to leverage all the advantages of a full binary Neural Network. Therefore, we obtain a deterministic binary Neural Network from the parameter distribution q θ (B) at test time. We consider three approaches for obtaining a deterministic network: a deterministic network based on the mode of q θ (B) called PBNET-MAP, an ensemble of binary Neural Networks sampled from q θ (B) named PBNET-x, and a ternary Neural Network (PBNET-TERNARY), in which a single parameter W i may be set to zero based on q θ, i.e.: DISPLAYFORM0 The ternary network can also be viewed as a sparse PBNet, however, sparse memory look-ups may slow down inference. Note that, even when using multiple binary neural networks in an ensemble, the ensemble is still more efficient in terms of computation and memory when compared to a full precision alternative. Moreover, it allows for anytime ensemble predictions for improved performance and uncertainty estimates by sampling from the weight distribution. Since the trained weight distribution is not fully deterministic, the sampling of individual weight instantiations will in a shift of the batch statistics. As a consequence, the learned batch norm statistics no longer closely match the true statistics. This is alleviated by re-estimating the batch norm statistics based on (a subset of) the training set after weight sampling using a moving mean and variance estimator. We observed competitive using as little as 20 batches from the training set. Binary and low precision neural networks have received significant interest in recent years. Most similar to our work, in terms of the final neural network, is the work on Binarized Neural Networks by BID7. in this work a real-valued shadow weight is used and binary weights are obtained by binarizing the shadow weights. Similarly the pre-activations are binarized using the same binarization function. In order to back-propagate through the binarization operation the straightthrough estimator BID6 ) is used. Several extensions to Binarized Neural Networks have been proposed which -more or less -qualify as binary neural networks: XNOR-net BID20 in which the real-valued parameter tensor and activation tensor is approximated by a binary tensor and a scaling factor per channel. ABC-nets take this approach one step further and approximate the weight tensor by a linear combination of binary tensors. Both of these approaches perform the linear operations in the forward pass using binary weights and/or binary activations, followed by a scaling or linear combination of the pre-activations. , similar methods to BID7 are used to binarize a wide resnet to obtain on ImageNet very close to the full precision performance. Another method for training binary neural networks is Expectation Backpropagation BID22 in which the central limit theorem and online expectation propagation is used to find an approximate posterior. This method is similar in spirit to ours, but the training method is completely different. Most related to our work is the work by BID21 which use the local reparametrization trick to train a Neural Network with binary weights and the work by BID1 which also discuss a binary Neural Network in which the activation distribution are propagated through the network. Moreover, in the CLT was used to approximate dropout noise during training in order to speed up training, however, there is no aim to learn binary (or discrete) weights or use binary activations in this work. We evaluate the PBNet on the MNIST and CIFAR-10 benchmarks and compare the to Binarized Neural Networks BID7, since the architectures of the deterministic networks obtained by training the PBNet are equivalent. The PBNets are trained using either a cross-entropy (CE) loss or a binary cross entropy for each class (BCE). For the CE loss there is no binarization step in the final layer, instead the mean of the Gaussian approximation is used as the input to a softmax layer. For BCE, there is a binarization step, and we treat the probability of the ith output being +1 as the probability of the input belonging to the ith class. Specifically, for an output vector p ∈ C for C classes and the true class y, the BCE loss for a single sample is defined as DISPLAYFORM0 The weights for the PBNet-S networks are initialized using the transfer method described in Section 2.3 and the PBNets are initialized using a uniform initialization scheme. All models are optimized using Adam and a validation loss plateau learning rate decay scheme. We keep the temperature for the binary concrete distribution static at 1.0 during training. For all settings, we optimize model parameters until convergence, after which the best model is selected based on a validation set. Our code is implemented using PyTorch BID19.For Binarized Neural Networks we use the training procedure described by BID7, i.e., a squared hinge loss and layer specific learning rates that are determined based on the Glorot initialization method BID4.Experimental details specific to datasets are given in Appendix C and the are presented in TAB0. We report both test set accuracy obtained after binarizing the network as well as the the test set accuracy obtained by the stochastic network during training (i.e., by propagating activation distributions). As presented in TAB0 the accuracy improves when using an ensemble. Moreover, the predictions of the ensemble members can be used to obtain an estimate of the certainty of the ensemble as a whole. BID7, PBNet, and a full precission network (FPNet). PBNet-map refers to a deterministic PBNet using the map estimate, PBNet-Ternary is a ternary deterministic network obtained from q θ, and PBNet-X refers to an ensemble of X networks, each sampled from the same weight distribution. For the ensemble both mean and standard deviation are presented. The propagate column contains obtained using the stochastic network whereas in the binarized column are obtained using a deterministic binary Neural Network. To evaluate this, we plot an error-coverage curve BID3 in FIG2. This curve is obtained by sorting the samples according to a statistic and computing the error percentage in the top x% of the samples -according to the statistic. For the Binarized Neural Network and PBNet-MAP the highest softmax score is used, whereas for the ensembles the variance in the prediction of the top class is used. The figure suggests that the ensemble variance is a better estimator of network certainty, and moreover, the estimation improves as the ensemble sizes increases. As discussed in Section 2.4, after sampling the parameters of a deterministic network the batch statistics used by Batch Normalization must be re-estimated. FIG2 shows the obtained using a various number of batches from the training set to re-estimate the statistics. This shows that even a small number of samples is sufficient to estimate the statistics. We perform an ablation study on both the use of (stochastic) Batch Normalization and the use of weight transfer for the PBNet-S on CIFAR-10. For Batch Normalization, we removed all batch normalization layers from the PBNet-S and retrained the model on CIFAR-10. This ed in a test set accuracy of 79.21%. For the weight initialization experiment, the PBNet-S weights are initialized using a uniform initialization scheme and is trained on CIFAR-10, ing in a test set accuracy of 83.61%. Moreover, the accuracy on the validation set during training is presented in FIG2. Note that these numbers are obtained without sampling a binarized network from the weight distribution, i.e., local reparametrization and binary activation samples are used. The PBNet-S that uses both weight transfer and stochastic Batch Normalization in a significant performance improvement, indicating that both stochastic Batch Normalization and weight transfer are necessary components for the PBNet-S. The of our experiments show that, following our training procedure, sampling of the binary activations is a necessary component. Although the stochastic PBNet generalizes well to unseen data, there is a significant drop in test accuracy when a binary Neural Network is obtained from the stochastic PBNet. In contrast, this performance drop is not observed for PBNet-S. A potential explanation of this phenomenon is that by sampling the binary activation during training, the network is forced to become more robust to the inherent binarization noise that is present at test time of the binarized Neural Network. If this is the case, then sampling the binary activation can be thought of as a regularization strategy that prepares the network for a more noisy binary setting. However, other regularization strategies may also exist. We have presented a stochastic method for training Binary Neural Networks. The method is evaluated on multiple standardized benchmarks and reached competitive . The PBNet has various advantageous properties as a of the training method. The weight distribution allows one to generate ensembles online which in improved accuracy and better uncertainty estimations. Moreover, the Bayesian formulation of the PBNet allows for further pruning of the network, which we leave as future work. A BINARY DISTRIBUTION For convenience, we have introduced the Binary distribution in this paper. In this appendix we list some of the properties used in the paper, which all follow direcly from the properties of the Bernoulli distribution. The Binary distribution is a reparametrization of the Bernoulli distribution such that: DISPLAYFORM0 This gives the following probability mass function: DISPLAYFORM1 where a ∈ {−1, +1} and θ ∈ [−1, 1]. From this, the mean and variance are easily computed: DISPLAYFORM2 Finally, let b ∼ Binary(φ), then ab ∼ Binary(θφ). During training the PBNet is trained using stochastic Batch Normalization. At test time, the parameters learned using stochastic Batch Normalization can be transferred to a conventional Batch Normalization implementation. Alternatively, Batch Normalization can be reduced to an (integer) addition and multiplication by ±1 after applying the sign activation function. Given a pre-activation a, the application of Batch Normalization followed by a sign binarization function can be rewritten as: DISPLAYFORM0 DISPLAYFORM1 when a ∈ Z, which is the case for all but the first layer Note that we have used sign = b det = +1 here, as we have used everywhere in order to use sign as a binarization function. The MNIST dataset consists of of 60K training and 10K test 28×28 grayscale handwritten digit images, divided over 10 classes. The images are pre-processed by subtracting the global pixel mean and dividing by the global pixel standard deviation. No other form of pre-processing or data augmentation is used. For MNIST, we use the following architecture: DISPLAYFORM0 where XC3 denotes a binary convolutional layer using 3 × 3 filters and X output channels, Y FC denotes a fully connected layer with Y output neurons, SM10 denotes a softmax layer with 10 outputs, and MP2 denotes 2 × 2 (stochastic) max pooling with stride 2. Note that if a convolutional layer is followed by a max pooling layer, the binarization is only performed after max pooling. All layers are followed by (stochastic) batch normalization and binarization of the activations. We use a batchsize of 128 and an initial learning rate of 10 −2 Results are reported in TAB0. The CIFAR-10 (BID12) dataset consists of 50K training and 10K test 32 × 32 RGB images divided over 10 classes. The last 5,000 images from the training set are used as validation set. Tthe images are only pre-processed by subtracting the channel-wise mean and dividing by the standard deviation. We use the following architecture for our CIFAR-10 experiment (following BID21): DISPLAYFORM0 where we use the same notation as in the previous section. The Binarized Neural Network baseline uses the same architecture, except for one extra 1024 neuron fully connected layer. During training, the training set is augmented using random 0px to 4px translations and random horizontal fl Results are reported in TAB0., a positive would be We introduce a stochastic training method for training Binary Neural Network with both binary weights and activations. & a negative would be This paper 1) characterizes functions representable by ReLU DNNs, 2) formally studies the benefit of depth in such architectures,  3) gives an algorithm to implement empirical risk minimization to global optimality for two layer ReLU nets.

Question: Generative Adversarial Networks (GANs) have recently achieved impressive for many real-world applications, and many GAN variants have emerged with improvements in sample quality and training stability. However, visualization and understanding of GANs is largely missing. How does a GAN represent our visual world internally? What causes the artifacts in GAN ? How do architectural choices affect GAN learning? Answering such questions could enable us to develop new insights and better models. In this work, we present an analytic framework to visualize and understand GANs at the unit-, object-, and scene-level. We first identify a group of interpretable units that are closely related to object concepts with a segmentation-based network dissection method. Then, we quantify the causal effect of interpretable units by measuring the ability of interventions to control objects in the output. Finally, we examine the contextual relationship between these units and their surrounding by inserting the discovered object concepts into new images. We show several practical applications enabled by our framework, from comparing internal representations across different layers, models, and datasets, to improving GANs by locating and removing artifact-causing units, to interactively manipulating objects in the scene. We provide open source interpretation tools to help peer researchers and practitioners better understand their GAN models. Generative Adversarial Networks (GANs) BID11 have been able to produce photorealistic images, often indistinguishable from real images. This remarkable ability has powered many real-world applications ranging from visual recognition BID35, to image manipulation, to video prediction. Since their invention in 2014, many GAN variants have been proposed BID29 BID41, often producing more realistic and diverse samples with better training stability. Despite this tremendous success, many questions remain to be answered. For example, to produce a church image (Figure 1a), what knowledge does a GAN need to learn? Alternatively, when a GAN sometimes produces terribly unrealistic images (Figure 1f), what causes the mistakes? Why does one GAN variant work better than another? What fundamental differences are encoded in their weights?In this work, we study the internal representations of GANs. To a human observer, a well-trained GAN appears to have learned facts about the objects in the image: for example, a door can appear on a building but not on a tree. We wish to understand how a GAN represents such structure. Do the objects emerge as pure pixel patterns without any explicit representation of objects such as doors and trees, or does the GAN contain internal variables that correspond to the objects that humans perceive? If the GAN does contain variables for doors and trees, do those variables cause the generation of those objects, or do they merely correlate? How are relationships between objects represented? Figure 1: Overview: (a) Realistic outdoor church images generated by Progressive GANs BID18. (b) Given a pre-trained GAN model, we identify a set of interpretable units whose featuremap is correlated to an object class across different images. For example, one unit in layer4 localizes tree regions with diverse visual appearance. (c) We force the activation of the units to be zero and quantify the average casual effect of the ablation. Here we successfully remove trees from church images. (d) We activate tree causal units in other locations. These same units synthesize new trees, visually compatible with their surrounding context. In addition, our method can diagnose and improve GANs by identifying artifact-causing units (e). We can remove the artifacts that appear (f) and significantly improve the by ablating the "artifact" units (g). Please see our demo video. We present a general method for visualizing and understanding GANs at different levels of abstraction, from each neuron, to each object, to the contextual relationship between different objects. We first identify a group of interpretable units that are related to object concepts (Figure 1b). These units' featuremaps closely match the semantic segmentation of a particular object class (e.g., trees). Second, we directly intervene within the network to identify sets of units that cause a type of objects to disappear (Figure 1c) or appear (Figure 1d). We quantify the causal effect of these units using a standard causality metric. Finally, we examine the contextual relationship between these causal object units and the . We study where we can insert object concepts in new images and how this intervention interacts with other objects in the image (Figure 1d). To our knowledge, our work provides the first systematic analysis for understanding the internal representations of GANs. Finally, we show several practical applications enabled by this analytic framework, from comparing internal representations across different layers, GAN variants and datasets; to debugging and improving GANs by locating and ablating "artifact" units (Figure 1e); to understanding contextual relationships between objects in scenes; to manipulating images with interactive object-level control. Generative Adversarial Networks. The quality and diversity of from GANs BID11 has continued to improve, from generating simple digits and faces BID11, to synthesizing natural scene images BID29 BID6, to generating 1k photorealistic portraits BID18, to producing one thousand object classes BID24 BID41. GANs have also enabled applications such as visual recognition BID35 BID14, image manipulation, and video generation BID34. Despite the successes, little work has been done to visualize what GANs have learned. Prior work BID29 BID48 BID4 manipulates latent vectors and observes how the change accordingly. Measuring the relationship between representation units and trees in the output using (a) dissection and (b) intervention. Dissection measures agreement between a unit u and a concept c by comparing its thresholded upsampled heatmap with a semantic segmentation of the generated image s c (x). Intervention measures the causal effect of a set of units U on a concept c by comparing the effect of forcing these units on (unit insertion) and off (unit ablation). The segmentation s c reveals that trees increase after insertion and decrease after ablation. The average difference in the tree pixels measures the average causal effect. In this figure, interventions are applied to the entire featuremap P, but insertions and ablations can also apply to any subset of pixels P ⊂ P. DISPLAYFORM0 Visualizing deep neural networks. A CNN can be visualized by reconstructing salient image features BID31 BID22 or by mining patches that maximize hidden layers' activations BID40 ); or we can synthesize input images to invert a feature layer BID9. Alternately, we can identify the semantics of each unit BID43 BID1 BID45 by measuring agreement between unit activations and object segmentation masks, or by training a network to increase interpretability of such units BID42. Visualization of an RNN has also revealed interpretable units that track long-range dependencies BID17 BID32. Most previous work on network visualization has focused on networks trained for classification; our work explores deep generative models trained for image generation. Understanding neural representation in biology. Studies of biological neural networks find evidence of both local representations in which individual neurons are selective for meaningful concepts , as well as distributed representations in which individual neurons are essentially meaningless BID39. Computational models of biological learning BID3 BID5 find sparse and local representations can aid generalization to novel stimuli. Explaining the decisions of deep neural networks. Individual network decisions can be explained using informative heatmaps BID46 BID30 or by scoring salience BID31 BID0 BID33 BID21. Such analyses reveals which inputs contribute most to a categorical prediction by a network. Recent work has also studied the contribution of feature vectors BID19 BID46 or individual channels BID26 to a final prediction, and BID25 has examined the effect of individual units by ablating them. Those methods explain discriminative classifiers. Our method aims to explain how an image can be generated by a network, which is much less explored. Our goal is to analyze how objects such as trees are encoded by the internal representations of a GAN generator G: z → x. Here z ∈ R |z| denotes a latent vector sampled from a low-dimensional distribution, and x ∈ R H×W ×3 denotes an H × W generated image. We use representation to Thresholding unit #65 layer 3 of a dining room generator matches'table' segmentations with IoU=0.34.Thresholding unit #37 layer 4 of a living room generator matches'sofa' segmentations with IoU=0.29.Figure 3: Visualizing the activations of individual units in two GANs. Top ten activating images are shown, and IoU is measured over a sample of 1000 images. In each image, the unit feature is upsampled and thresholded as described in Eqn. 2.describe the tensor r output from a particular layer of the generator G, where the generator creates an image x from random z through a composition of layers: r = h(z) and DISPLAYFORM0 Since r has all the data necessary to produce the image x = f (r), r certainly contains the information to deduce the presence of any visible class c in the image. Therefore the question we ask is not whether information about c is present in r -it is -but how such information is encoded in r. In particular, for any class from a universe of concepts c ∈ C, we seek to understand whether r explicitly represents c in some way where it is possible to factor r at locations P into two components r U,P = (r U,P, r U,P),where the generation of the object c at locations P depends mainly on the units r U,P, and is insensitive to the other units r U,P. Here we refer to each channel of the featuremap as a unit: U denotes the set of unit indices of interest and U is its complement; we will write U and P to refer to the entire set of units and featuremap pixels in r. We study the structure of r in two phases:• Dissection: starting with a large dictionary of object classes, we identify the classes that have an explicit representation in r by measuring the agreement between individual units of r and every class c (Figure 1b).• Intervention: for the represented classes identified through dissection, we identify causal sets of units and measure causal effects between units and object classes by forcing sets of units on and off (Figure 1c,d). We first focus on individual units of the representation. Recall that r u,P is the one-channel h × w featuremap of unit u in a convolutional generator, where h × w is typically smaller than the image size. We want to know if a specific unit r u,P encodes a semantic class such as a "tree". For image classification networks, BID1 has observed that many units can approximately locate emergent object classes when the units are upsampled and thresholded. In that spirit, we select a universe of concepts c ∈ C for which we have a semantic segmentation s c (x) for each class. Then we quantify the spatial agreement between the unit u's thresholded featuremap and a concept c's segmentation with the following intersection-over-union (IoU) measure: DISPLAYFORM0, where t u,c = arg max DISPLAYFORM1 where ∧ and ∨ denote intersection and union operations, and x = G(z) denotes the image generated from z. The one-channel feature map r u,P slices the entire featuremap r = h(z) at unit u. As shown in FIG1, we upsample r u,P to the output image resolution as r ↑ u,P. (r ↑ u,P > t u,c) produces a binary mask by thresholding the r ↑ u,P at a fixed level t u,c. s c (x) is a binary mask where each pixel indicates the presence of class c in the generated image x. The threshold t u,c is chosen to be informative as possible by maximizing the information quality ratio I/H (using a separate validation set), that is, it maximizes the portion of the joint entropy H which is mutual information I BID36.We can use IoU u,c to rank the concepts related to each unit and label each unit with the concept that matches it best. Figure 3 shows examples of interpretable units with high IoU u,c. They are not the Figure 4: Ablating successively larger sets of tree-causal units from a GAN trained on LSUN outdoor church images, showing that the more units are removed, the more trees are reduced, while buildings remain. The choice of units to ablate is specific to the tree class and does not depend on the image. At right, the causal effect of removing successively more tree units is plotted, comparing units chosen to optimize the average causal effect (ACE) and units chosen with the highest IoU for trees.only units to match tables and sofas: layer3 of the dining room generator has 31 units (of 512) that match tables and table parts, and layer4 of the living room generator has 65 (of 512) sofa units. Once we have identified an object class that a set of units match closely, we next ask: which units are responsible for triggering the rendering of that object? A unit that correlates highly with an output object might not actually cause that output. Furthermore, any output will jointly depend on several parts of the representation. We need a way to identify combinations of units that cause an object. To answer the above question about causality, we probe the network using interventions: we test whether a set of units U in r cause the generation of c by forcing the units of U on and off. Recall that r U,P denotes the featuremap r at units U and locations P. We ablate those units by forcing r U,P = 0. Similarly, we insert those units by forcing r U,P = k, where k is a per-class constant, as described in Section S-6.4. We decompose the featuremap r into two parts (r U,P, r U,P), where r U,P are unforced components of r:Original image: DISPLAYFORM0 Image with U ablated at pixels P: DISPLAYFORM1 Image with U inserted at pixels P: DISPLAYFORM2 An object is caused by U if the object appears in x i and disappears from x a. Figure 1c demonstrates the ablation of units that remove trees, and Figure 1d demonstrates insertion of units at specific locations to make trees appear. This causality can be quantified by comparing the presence of trees in x i and x a and averaging effects over all locations and images. Following prior work BID15 BID27, we define the average causal effect (ACE) of units U on the generation of on class c as: DISPLAYFORM3 where s c (x) denotes a segmentation indicating the presence of class c in the image x at P. To permit comparisons of δ U→c between classes c which are rare, we normalize our segmentation s c by DISPLAYFORM4. While these measures can be applied to a single unit, we have found that objects tend to depend on more than one unit. Thus we wish to identify a set of units U that maximize the average causal effect δ U→c for an object class c. Finding sets of units with high ACE. Given a representation r with d units, exhaustively searching for a fixed-size set U with high δ U→c is prohibitive as it has d |U| subsets. Instead, we optimize a continuous intervention α ∈ d, where each dimension α u indicates the degree of intervention for a unit u. We maximize the following average causal effect formulation δ α→c: Image with partial ablation at pixels P: DISPLAYFORM5 Image with partial insertion at pixels P: DISPLAYFORM6 Objective: DISPLAYFORM7 where r U,P denotes the all-channel featuremap at locations P, r U,P denotes the all-channel featuremap at other locations P, and applies a per-channel scaling vector α to the featuremap r U,P. We optimize α over the following loss with an L2 regularization: DISPLAYFORM8 where λ controls the relative importance of each term. We add the L2 loss as we seek a minimal set of casual units. We optimize using stochastic gradient descent, sampling over both z and featuremap locations P and clamping the coefficient α within the range d at each step (d is the total number of units). More details of this optimization are discussed in Section S-6.4. Finally, we can rank units by α * u and achieve a stronger causal effect (i.e., removing trees) when ablating successively larger sets of tree-causing units as shown in Figure 4. We study three variants of Progressive GANs BID18 ) trained on LSUN scene datasets BID38. To segment the generated images, we use a recent model BID37 trained on the ADE20K scene dataset. The model can segment the input image into 336 object classes, 29 parts of large objects, and 25 materials. To further identify units that specialize in object parts, we expand each object class c into additional object part classes c-t, c-b, c-l, and c-r, which denote the top, bottom, left, or right half of the bounding box of a connected component. Below, we use dissection for analyzing and comparing units across datasets, layers, and models (Section 4.1), and locating artifact units (Section 4.2). Then, we start with a set of dominant object classes and use intervention to locate causal units that can remove and insert objects in different images (Section 4.3 and 4.4). In addition, our video demonstrates our interactive tool. Emergence of individual unit object detectors We are particularly interested in any units that are correlated with instances of an object class with diverse visual appearances; these would suggest that GANs generate those objects using similar abstractions as humans. Figure 3 illustrates two such units. In the dining room dataset, a unit emerges to match dining table regions. More interestingly, the matched tables have different colors, materials, geometry, viewpoints, and levels of clutter: the only obvious commonality among these regions is the concept of a table. This unit's featuremap correlates to the fully supervised segmentation model BID37 ) with a high IoU of 0.34.Interpretable units for different scene categories The set of all object classes matched by the units of a GAN provides a map of what a GAN has learned about the data. FIG3 examines units from GANs trained on four LSUN scene categories BID38. The units that emerge are object classes appropriate to the scene type: for example, when we examine a GAN trained on kitchen scenes, we find units that match stoves, cabinets, and the legs of tall kitchen stools. Another striking phenomenon is that many units represent parts of objects: for example, the conference room GAN contains separate units for the body and head of a person. Interpretable units for different network layers. In classifier networks, the type of information explicitly represented changes from layer to layer BID40. We find a similar phenomenon in a GAN. FIG4 compares early, middle, and late layers of a progressive GAN with 14 internal convolutional layers. The output of the first convolutional layer, one step away from the input z, remains entangled: individual units do not correlate well with any object classes except for two units that are biased towards the ceiling of the room. Mid-level layers 4 to 7 have many units that match semantic objects and object parts. Units in layers 10 and beyond match local pixel patterns such as materials, edges and colors. All layers are shown in Section S-6.7.Interpretable units for different GAN models. Interpretable units can provide insights about how GAN architecture choices affect the structures learned inside a GAN. FIG5 compares three models from BID18: a baseline Progressive GANs, a modification that introduces minibatch stddev statistics, and a further modification that adds pixelwise normalization. By examining unit semantics, we confirm that providing minibatch stddev statistics to the discriminator increases not only the realism of , but also the diversity of concepts represented by units: the number of types of objects, parts, and materials matching units increases by more than 40%. The pixelwise normalization increases the number of units that match semantic classes by 19%. The output of the first convolutional layer has almost no units that match semantic objects, but many objects emerge at layers 4-7. Later layers are dominated by low-level materials, edges and colors. While our framework can reveal how GANs succeed in producing realistic images, it can also analyze the causes of failures in their . FIG6 shows several annotated units that are responsible for typical artifacts consistently appearing across different images. We can identify these units efficiently by human annotation: out of a sample of 1000 images, we visualize the top ten highest activating images for each unit, and we manually identify units with noticeable artifacts in this set. It typically takes 10 minutes to locate 20 artifact-causing units out of 512 units in layer4.More importantly, we can fix these errors by ablating the above 20 artifact-causing units. FIG6 shows that artifacts are successfully removed, and the artifact-free pixels stay the same, improving the generated . In TAB3 we report two standard metrics, comparing our improved images to both the original artifact images and a simple baseline that ablates 20 randomly chosen units. First, we compute the widely used Fréchet Inception Distance BID13 between the generated images and real images. We use 50, 000 real images and generate 10, 000 images with high Note that as the quality of the model improves, the number of interpretable units also rises. Progressive GANs apply several innovations including making the discriminator aware of minibatch statistics, and pixelwise normalization at each layer. We can see batch awareness increases the number of object classes matched by units, and pixel norm (applied in addition to batch stddev) increases the number of units matching objects. activations on these units. Second, we score 1, 000 images per method on Amazon MTurk, collecting 20, 000 human annotations regarding whether the modified image looks more realistic compared to the original. Both metrics show significant improvements. Strikingly, this simple manual change to a network beats state-of-the-art GANs models. The manual identification of "artifact" units can be approximated by an automatic scoring of the realism of each unit, as detailed in Section S-6.1. Errors are not the only type of output that can be affected by directly intervening in a GAN. A variety of specific object types can also be removed from GAN output by ablating a set of units in a GAN. In Figure 9 we apply the method in Section 3.2 to identify sets of 20 units that have causal effects on common object classes in conference rooms scenes. We find that, by turning off these small sets of units, most of the output of people, curtains, and windows can be removed from the generated scenes. However, not every object can be erased: tables and chairs cannot be removed. Ablating those units will reduce the size and density of these objects, but will rarely eliminate them. The ease of object removal depends on the scene type. Figure 10 shows that, while windows can be removed well from conference rooms, they are more difficult to remove from other scenes. In particular, windows are just as difficult to remove from a bedroom as tables and chairs from a conference room. We hypothesize that the difficulty of removal reflects the level of choice that a GAN has learned for a concept: a conference room is defined by the presence of chairs, so they cannot be altered. And modern building codes mandate that all bedrooms must have windows; the GAN seems to have caught on to that pattern. Figure 9: Measuring the effect of ablating units in a GAN trained on conference room images. Five different sets of units have been ablated related to a specific object class. In each case, 20 (out of 512) units are ablated from the same GAN model. The 20 units are specific to the object class and independent of the image. The average causal effect is reported as the portion of pixels that are removed in 1 000 randomly generated images. We observe that some object classes are easier to remove cleanly than others: a small ablation can erase most pixels for people, curtains, and windows, whereas a similar ablation for tables and chairs only reduces object sizes without deleting them. We can also learn about the operation of a GAN by forcing units on and inserting these features into specific locations in scenes. Figure 11 shows the effect of inserting 20 layer4 causal door units in church scenes. In this experiment, we insert these units by setting their activation to the fixed mean value for doors (further details in Section S-6.4). Although this intervention is the same in each case, the effects vary widely depending on the objects' surrounding context. For example, the doors added to the five buildings in Figure 11 appear with a diversity of visual attributes, each with an orientation, size, material, and style that matches the building. We also observe that doors cannot be added in most locations. The locations where a door can be added are highlighted by a yellow box. The bar chart in Figure 11 shows average causal effects of insertions of door units, conditioned on the object class at the location of the intervention. We find that the GAN allows doors to be added in buildings, particularly in plausible locations such as where a window is present, or where bricks are present. Conversely, it is not possible to trigger a door in the sky or on trees. Interventions provide insight on how a GAN enforces relationships between objects. Even if we try to add a door in layer4, that choice can be vetoed later if the object is not appropriate for the context. These downstream effects are further explored in Section S-6.5. By carefully examining representation units, we have found that many parts of GAN representations can be interpreted, not only as signals that correlate with object concepts but as variables that have a causal effect on the synthesis of objects in the output. These interpretable effects can be used to compare, debug, modify, and reason about a GAN model. Our method can be potentially applied to other generative models such as VAEs BID20 and RealNVP BID7.We have focused on the generator rather than the discriminator (as did in BID29) because the generator must represent all the information necessary to approximate the target distribution, while the discriminator only learns to capture the difference between real and fake images. Alternatively, we conference room church living room kitchen bedroom Figure 10: Comparing the effect of ablating 20 window-causal units in GANs trained on five scene categories. In each case, the 20 ablated units are specific to the class and the generator and independent of the image. In some scenes, windows are reduced in size or number rather than eliminated, or replaced by visually similar objects such as paintings. DISPLAYFORM0 Figure 11: Inserting door units by setting 20 causal units to a fixed high value at one pixel in the representation. Whether the door units can cause the generation of doors is dependent on its local context: we highlight every location that is responsive to insertions of door units on top of the original image, including two separate locations in (b) (we intervene at left). The same units are inserted in every case, but the door that appears has a size, alignment, and color appropriate to the location. Emphasizing a door that is already present in a larger door (d). The chart summarizes the causal effect of inserting door units at one pixel with different contexts.can train an encoder to invert the generator BID8. However, this incurs additional complexity and errors. Many GANs also do not have an encoder. Our method is not designed to compare the quality of GANs to one another, and it is not intended as a replacement for well-studied GAN metrics such as FID, which estimate realism by measuring the distance between the generated distribution of images and the true distribution BID2 surveys these methods). Instead, our goal has been to identify the interpretable structure and provide a window into the internal mechanisms of a GAN.Prior visualization methods BID40 BID1 BID17 have brought new insights into CNN and RNN research. Motivated by that, in this work we have taken a small step towards understanding the internal representations of a GAN, and we have uncovered many questions that we cannot yet answer with the current method. For example: why can a door not be inserted in the sky? How does the GAN suppress the signal in the later layers? Further work will be needed to understand the relationships between layers of a GAN. Nevertheless, we hope that our work can help researchers and practitioners better analyze and develop their own GANs. In Section 4.2, we have improved GANs by manually identifying and ablating artifact-causing units. Now we describe an automatic procedure to identify artifact units using unit-specific FID scores. To compute the FID score BID13 for a unit u, we generate 200, 000 images and select the 10, 000 images that maximize the activation of unit u, and this subset of 10, 000 images is compared to the true distribution (50, 000 real images) using FID. Although every such unit-maximizing subset of images represents a skewed distribution, we find that the per-unit FID scores fall in a wide range, with most units scoring well in FID while a few units stand out with bad FID scores: many of them were also manually flagged by humans, as they tend to activate on images with clear visible artifacts. FIG1 shows the performance of FID scores as a predictor of manually flagged artifact units. The per-unit FID scores can achieve 50% precision and 50% recall. That is, of the 20 worst-FID units, 10 are also among the 20 units manually judged to have the most noticeable artifacts. Furthermore, repairing the model by ablating the highest-FID units works: qualitative are shown in FIG8 and quantitative are shown in TAB4. (a) unit118 in layer4 DISPLAYFORM1 Figure 14: Two examples of generator units that our dissection method labels differently from humans. Both units are taken from layer4 of a Progressive GAN of living room model. In (a), human label the unit as'sofa' based on viewing the top-20 activating images, and our method labels as'ceiling'. In this case, our method counts many ceiling activations in a sample of 1000 images beyond the top 20. In (b), the dissection method has no confident label prediction even though the unit consistently triggers on white letterbox shapes at the top and bottom of the image. The segmentation model we use has no label for such abstract shapes. As a sanity check, we evaluate the gap between human labeling of object concepts correlated with units and our automatic segmentation-based labeling, for one model, as follows. For each of 512 units of layer4 of a "living room" Progressive GAN, 5 to 9 human annotations were collected (3728 labels in total). In each case, an AMT worker is asked to provide one or two words describing the highlighted patches in a set of top-activating images for a unit. Of the 512 units, 201 units were described by the same consistent word (such as "sofa", "fireplace" or "wicker") in 50% or more of the human labels. These units are interpretable to humans. Applying our segmentation-based dissection method, 154/201 of these units are also labeled with a confident label with IoU > 0.05 by dissection. In 104/154 cases, the segmentation-based model gave the same label word as the human annotators, and most others are slight shifts in specificity. For example, the segmentation labels "ottoman" or "curtain" or "painting" when a person labels "sofa" or "window" or "picture," respectively. A second AMT evaluation was done to rate the accuracy of both segmentation-derived and human-derived labels. Human-derived labels scored 100% (of the 201 human-labeled units, all of the labels were rated as consistent by most raters). Of the 154 segmentation-generated labels, 149 (96%) were rated by most AMT raters as accurate as well. The five failure cases (where the segmentation is confident but rated as inaccurate by humans) arise from situations in which human evaluators saw one concept after observing only 20 top-activating images, while the algorithm, in evaluating 1000 images, counted a different concept as dominant. Figure 14a shows one example: in the top images, mostly sofas are highlighted and few ceilings, whereas in the larger sample, mostly ceilings are triggered. There are also 47/201 cases where the segmenter is not confident while humans have consensus. Some of these are due to missing concepts in the segmenter. Figure 14b shows a typical example, where a unit is devoted to letterboxing (white stripes at the top and bottom of images), but the segmentation has no confident label to assign to these. We expect that as future semantic segmentation models are developed to be able to identify more concepts such as abstract shapes, more of these units can be automatically identified. Our method relies on having a segmentation function s c (x) that identifies pixels of class c in the output x. However, the segmentation model s c can perform poorly in the cases where x does not resemble the original training set of s c. This phenomenon is visible when analyzing earlier GAN models. For example, FIG3 visualizes two units from a WGAN-GP model BID12 for LSUN bedrooms (this model was trained by BID18 as a baseline in the original paper). For these two units, the segmentation network seems to be confused by the distorted images. To protect against such spurious segmentation labels, we can use a technique similar to that described in Section S-6.1: automatically identify units that produce unrealistic images, and omit those "unrealistic" units from semantic segmentation. An appropriate threshold to apply will depend on the distribution being modeled: in FIG4, we show how applying a filter, ignoring segmentation on units with FID 55 or higher, affects the analysis of this base WGAN model. In general, fewer irrelevant labels are associated with units. In this section we provide more details about the ACE optimization described in Section 3.2.Specifying the per-class positive intervention constant k. In Eqn. 3, the negative intervention is defined as zeroing the intervened units, and a positive intervention is defined as setting the intervened units to some big class-specific constant k. For interventions for class c, we set k to be mean featuremap activation conditioned on the presence of class c at that location in the output, with each pixel weighted by the portion of the featuremap locations that are covered by the class c. Setting all units at a pixel to k will tend to strongly cause the target class. The goal of the optimization is to find the subset of units that is causal for c. Sampling c-relevant locations P. When optimizing the causal objective (Eqn. 5), the intervention locations P are sampled from individual featuremap locations. When the class c is rare, most featuremap locations are uninformative: for example, when class c is a door in church scenes, most regions of the sky, grass, and trees are locations where doors will not appear. Therefore, we focus the optimization as follows: during training, minibatches are formed by sampling locations P that are An identical "door" intervention at layer4 of each pixel in the featuremap has a different effect on later feature layers, depending on the location of the intervention. In the heatmap, brighter colors indicate a stronger effect on the layer14 feature. A request for a door has a larger effect in locations of a building, and a smaller effect near trees and sky. At right, the magnitude of feature effects at every layer is shown, measured by the changes of mean-normalized features. In the line plot, feature changes for interventions that in human-visible changes are separated from interventions that do not in noticeable changes in the output.relevant to class c by including locations where the class c is present in the output (and are therefore candidates for removal by ablating a subset of units), and an equal portion of locations where class c is not present at P, but it would be present if all the units are set to the constant k (candidate locations for insertion with a subset of units). During the evaluation, causal effects are evaluated using uniform samples: the region P is set to the entire image when measuring ablations, and to uniformly sampled pixels P when measuring single-pixel insertions. Initializing α with IoU. When optimizing causal α for class c, we initialize with DISPLAYFORM0 That is, we set the initial α so that the largest component corresponds to the unit with the largest IoU for class c, and we normalize the components so that this largest component is 1.Applying a learned intervention α When applying the interventions, we clip α by keeping only its top n components and zeroing the remainder. To compare the interventions of different classes an different models on an equal basis, we examine interventions where we set n = 20. To investigate the mechanism for suppressing the visible effects of some interventions seen in Section 4.4, in this section we insert 20 door-causal units on a sample of individual featuremap locations at layer4 and measure the changes caused in later layers. To quantify effects on downstream features, the change in each feature channel is normalized by that channel's mean L1 magnitude, and we examine the mean change in these normalized featuremaps at each layer. In FIG5, these effects that propagate to layer14 are visualized as a heatmap: brighter colors indicate a stronger effect on the final feature layer when the door intervention is in the neighborhood of a building instead of trees or sky. Furthermore, we plot the average effect on every layer at right in FIG5, separating interventions that have a visible effect from those that do not. A small identical intervention at layer4 is amplified to larger changes up to a peak at layer12. Dissection can also be used to monitor the progress of training by quantifying the emergence, diversity, and quality of interpretable units. For example, in FIG6 we show dissections of layer4 representations of a Progressive GAN model trained on bedrooms, captured at a sequence of checkpoints during training. As training proceeds, the number of units matching objects increases, as does the number of object classes with matching units, and the quality of object detectors as The number and quality of interpretable units increases during training. Note that in early iterations, Progressive GAN generates images at a low resolution. The top-activating images for the same four selected units is shown for each iteration, along with the IoU and the matched concept for each unit at that checkpoint. measured by average IoU over units increases. During this successful training, dissection suggests that the model is gradually learning the structure of a bedroom, as increasingly units converge to meaningful bedroom concepts. In Section 4.1 we show a small selection of layers of a GAN; in Figure 19 we show a complete listing of all the internal convolutional layers of that model (a Progressive GAN trained on LSUN living room images). As can be seen, the diversity of units matching high-level object concepts peaks at layer4-layer6, then declines in later layers, with the later layers dominated by textures, colors, and shapes.

Represent this text to find a much much longer article about it with say hundreds-thousands of words.

GAN representations are examined in detail, and sets of representation units are found that control the generation of semantic concepts in the output.

Represent the TLDR for finding the corresponding scientific article

Conditional VAE on top of latent spaces of pre-trained generative models that enables  transfer between drastically different domains while preserving locality and semantic alignment.