Question: Represent the article for finding a single-sentence TLDR.

Question: Likelihood-based generative models are a promising resource to detect out-of-distribution (OOD) inputs which could compromise the robustness or reliability of a machine learning system. However, likelihoods derived from such models have been shown to be problematic for detecting certain types of inputs that significantly differ from training data. In this paper, we pose that this problem is due to the excessive influence that input complexity has in generative models' likelihoods. We report a set of experiments supporting this hypothesis, and use an estimate of input complexity to derive an efficient and parameter-free OOD score, which can be seen as a likelihood-ratio, akin to Bayesian model comparison. We find such score to perform comparably to, or even better than, existing OOD detection approaches under a wide range of data sets, models, model sizes, and complexity estimates. Assessing whether input data is novel or significantly different than the one used in training is critical for real-world machine learning applications. Such data are known as out-of-distribution (OOD) inputs, and detecting them should facilitate safe and reliable model operation. This is particularly necessary for deep neural network classifiers, which can be easily fooled by OOD data . Several approaches have been proposed for OOD detection on top of or within a neural network classifier (; ; ;). Nonetheless, OOD detection is not limited to classification tasks nor to labeled data sets. Two examples of that are novelty detection from an unlabeled data set and next-frame prediction from video sequences. A rather obvious strategy to perform OOD detection in the absence of labels (and even in the presence of them) is to learn a density model M that approximates the true distribution p * (X) of training inputs x ∈ X . Then, if such approximation is good enough, that is, p(x|M) ≈ p * (x), OOD inputs should yield a low likelihood under model M. With complex data like audio or images, this strategy was long thought to be unattainable due to the difficulty of learning a sufficiently good model. However, with current approaches, we start having generative models that are able to learn good approximations of the density conveyed by those complex data. Autoregressive and invertible models such as PixelCNN++ and Glow perform well in this regard and, in addition, can approximate p(x|M) with arbitrary accuracy. Figure 1: Likelihoods from a Glow model trained on CIFAR10. Qualitatively similar are obtained for other generative models and data sets (see also in ; a). trained on CIFAR10, generative models report higher likelihoods for SVHN than for CIFAR10 itself (Fig. 1 ; data descriptions are available in Appendix A). Intriguingly, this behavior is not consistent across data sets, as other ones correctly tend to produce likelihoods lower than the ones of the training data (see the example of TrafficSign in Fig. 1). A number of explanations have been suggested for the root cause of this behavior (; a;) but, to date, a full understanding of the phenomenon remains elusive. In this paper, we shed light to the above phenomenon, showing that likelihoods computed from generative models exhibit a strong bias towards the complexity of the corresponding inputs. We find that qualitatively complex images tend to produce the lowest likelihoods, and that simple images always yield the highest ones. In fact, we show a clear negative correlation between quantitative estimates of complexity and the likelihood of generative models. In the second part of the paper, we propose to leverage such estimates of complexity to detect OOD inputs. To do so, we introduce a widely-applicable OOD score for individual inputs that corresponds, conceptually, to a likelihoodratio test statistic. We show that such score turns likelihood-based generative models into practical and effective OOD detectors, with performances comparable to, or even better than the state-of-theart. We base our experiments on an extensive collection of alternatives, including a pool of 12 data sets, two conceptually-different generative models, increasing model sizes, and three variants of complexity estimates. From now on, we shall consider the log-likelihood of an input x given a model M: M (x) = log 2 p(x|M). Following common practice in evaluating generative models, negative log-likelihoods − M will be expressed in bits per dimension , where dimension corresponds to the total size of x (we resize all images to 3×32×32 pixels). Note that the qualitative behavior of log-likelihoods is the same as likelihoods: ideally, OOD inputs should have a low M, while in-distribution data should have a larger M. Most literature compares likelihoods of a given model for a few data sets. However, if we consider several different data sets at once and study their likelihoods, we can get some insight. In Fig. 2, we show the log-likelihood distributions for the considered data sets (Appendix A), computed with a Glow model trained on CIFAR10. We observe that the data set with a higher log-likelihood is Constant, a data set of constant-color images, followed by Omniglot, MNIST, and FashionMNIST; all of those featuring gray-scale images with a large presence of empty black . On the other side of the spectrum, we observe that the data set with a lower log-likelihood is Noise, a data set of uniform random images, followed by TrafficSign and TinyImageNet; both featuring colorful images with non-trivial . Such ordering is perhaps more clear by looking at the average log-likelihood of each data set (Appendix D). If we think about the visual complexity of the images in those data sets, it would seem that log-likelihoods tend to grow when images become simpler and with less information or content. To further confirm the previous observation, we design a controlled experiment where we can set different decreasing levels of image complexity. We train a generative model with some data set, as before, but now compute likelihoods of progressively simpler inputs. Such inputs are obtained by average-pooling the uniform random Noise images by factors of 1, 2, 4, 8, 16, and 32, and re-scaling back the images to the original size by nearest-neighbor up-sampling. Intuitively, a noise image with a pooling size of 1 (no pooling) has the highest complexity, while a noise image with a pooling of 32 (constant-color image) has the lowest complexity. Pooling factors from 2 to 16 then account for intermediate, decreasing levels of complexity. The of the experiment is a progressive growing of the log-likelihood M (Fig. 3). Given that the only difference between data is the pooling factor, we can infer that image complexity plays a major role in generative models' likelihoods. Until now, we have consciously avoided a quantitative definition of complexity. However, to further study the observed phenomenon, and despite the difficulty in quantifying the multiple aspects that affect the complexity of an input (cf.), we have to adopt one. A sensible choice would be to exploit the notion of Kolmogorov complexity which, unfortunately, is noncomputable. In such cases, we have to deal with it by calculating an upper bound using a lossless compression algorithm . Given a set of inputs x coded with the same bit depth, the normalized size of their compressed versions, L(x) (in bits per dimension), can be considered a reasonable estimate of their complexity. That is, given the same coding depth, a highly complex input will require more bits per dimension, while a less complex one will be compressed with fewer bits per dimension. For images, we can use PNG, JPEG2000, or FLIF compressors (Appendix C). For other data types such as audio or text, other lossless compressors should be available to produce a similar estimate. If we study the relation between generative models' likelihoods and our complexity estimates, we observe that there is a clear negative correlation (Fig. 4). Considering all data sets, we find Pearson's correlation coefficients below −0.75 for models trained on FashionMNIST, and below −0.9 for models trained on CIFAR10, independently of the compressor used (Appendix D). Such significant correlations, all of them with infinitesimal p-values, indicate that likelihood-based measures are highly influenced by the complexity of the input image, and that this concept accounts for most of their variance. In fact, such strong correlations suggest that one may replace the computed likelihood values for the negative of the complexity estimate and obtain almost the same (Appendix D). This implies that, in terms of detecting OOD inputs, a complexity estimate would perform as well (or bad) as the likelihoods computed from our generative models. As complexity seems to account for most of the variability in generative models' likelihoods, we propose to compensate for it when testing for possible OOD inputs. Given that both negative loglikelihoods − M (x) and the complexity estimate L(x) are expressed in bits per dimension (Sec. 2), we can express our OOD score as a subtraction between the two: Notice that, since we use negative log-likelihoods, the higher the S, the more OOD the input x will be (see below). Interestingly, S can be interpreted as a likelihood-ratio test statistic. For that, we take the point of view of Bayesian model comparison or minimum description length principle . We can think of a compressor M 0 as a universal model, adjusted for all possible inputs and general enough so that it is not biased towards a particular type of data semantics. Considering the probabilistic model associated with the size of the output produced by the lossless compressor, we and, correspondingly, In Bayesian model comparison, we are interested in comparing the posterior probabilities of different models in light of data X. In our setting, the trained generative model M is a'simpler' version of the universal model M 0, targeted to a specific semantics or data type. With it, one aims to approximate the marginal likelihood (or model evidence) for x ∈ X, which integrates out all model parameters: This integral is intractable, but current generative models can approximate p(x|M) with arbitrary accuracy . Choosing between one or another model is then reduced to a simple likelihood ratio: For uniform priors p(M 0) = p(M) = 1/2, this ratio is reduced to which, using Eq. 2 for the last term, becomes Eq. 1. The ratio S accommodates the Occam's razor principle. Consider simple inputs that can be easily compressed by M 0 using a few bits, and that are not present in the training of M. These cases have a high probability under M 0, effectively correcting the abnormal high likelihood given by the learned model M. The same effect will occur with complex inputs that are not present in the training data. In these cases, both likelihoods will be low, but the universal lossless compressor M 0 will predict those better than the learned model M. The two situations will lead to large values of S. In contrast, inputs that belong to the data used to train the generative model M will always be better predicted by M than by M 0, ing in lower values of S. Given a training set X of in-distribution samples and the corresponding scores S(x) for each x ∈ X, we foresee a number of strategies to perform OOD detection in practice for new instances z. The first and more straightforward one is to use S(z) as it is, just as a score, to perform OOD ranking. This can be useful to monitor the top-k, potentially more problematic instances z in a new set of unlabeled data Z. The second strategy is to interpret S(z) as the corresponding Bayes factor in Eq. 3, and directly assign z to be OOD for S(z) > 0, or in-distribution otherwise (cf.). The decision is then taken with stronger evidence for higher absolute values of S(z). A third strategy is to consider the empirical or null distribution of S for the full training set, S(X). We could then choose an appropriate quantile as threshold, adopting the notion of frequentist hypothesis testing (see for instance b). Finally, if ground truth OOD data Y is available, a fourth strategy is to optimize a threshold value for S(z). Using X and Y, we can choose a threshold that targets a desired percentage of false positives or negatives. The choice of a specific strategy will depend on the characteristics of the particular application under consideration. In this work, we prefer to keep our evaluation generic and to not adopt any specific thresholding strategy (that is, we use S directly, as a score). This also allows us to compare with the majority of reported values from the existing literature, which use the AUROC measure (see below). 4 have recently proposed the use of likelihood-ratio tests for OOD detection. They posit that " statistics" (for instance, the number of zeros in the of MNISTlike images) are the source of abnormal likelihoods, and propose to exploit them by learning a model which is trained on random surrogates of input data. Such surrogates are generated according to a Bernoulli distribution, and an L2 regularization term is added to the model, which implies that the approach has two hyper-parameters. Moreover, both the model and the model trained using in-distribution data need to capture the information equally well. In contrast to their method, our method does not require additional training nor extra conditions on a specific model for every type of training data. and Nalisnick et al. (2019b) suggest that typicality is the culprit for likelihoodbased generative models not being able to detect OOD inputs. do not explicitly address typicality, their estimate of the Watanabe-Akaike information criterion using ensembles of generative models performs well in practice. Nalisnick et al. (2019b) propose an explicit test for typicality employing a Monte Carlo estimate of the empirical entropy, which limits their approach to batches of inputs of the same type. The works of Høst- and combine the concepts of typicality and minimum description length to perform novelty detection. Although concepts are similar to the ones employed here, their focus is mainly on bit sequences. They consider atypical sequences those that can be described (coded) with fewer bits in itself rather than using the (optimum) code for typical sequences. We find their implementation to rely on strong parametric assumptions, which makes it difficult to generalize to generative or other machine learning models. A number of methods have been proposed to perform OOD detection under a classification-based framework (; ; ; ;). Although achieving promising , these methods do not generally apply to the more general case of non-labeled or self-supervised data. The method of extends to such cases by leveraging generative models, but nonetheless makes use of auxiliary, outlier data to learn to distinguish OOD inputs. We now study how S performs on the OOD detection task. For that, we train a generative model M on the train partition of a given data set and compute scores for such partition and the test partition of a different data set. With both sets of scores, we then calculate the area under the receiver operating characteristic curve (AUROC), which is a common evaluation measure for the OOD detection task and for classification tasks in general. Note that AUROC represents a good performance summary across different score thresholds . First of all, we want to assess the improvement of S over log-likelihoods alone (− M). When considering likelihoods from generative models trained on CIFAR10, the problematic reported by previous works become clearly apparent (Table 1). The unintuitive higher likelihoods for SVHN observed in Sec. 1 now translate into a poor AUROC below 0.1. This not only happens for SVHN, but also for Constant, Omniglot, MNIST, and FashionMNIST data sets, for which we observed consistently higher likelihoods than CIFAR10 in Sec. 2. Likelihoods for the other data sets yield AUROCs above the random baseline of 0.5, but none above 0.67. The only exception is the Noise data set, which is perfectly distinguishable from CIFAR10 using likelihood alone. For completeness, we include the AUROC values when trying to perform OOD with the test partition of CIFAR10. We see those are close to the ideal value of 0.5, showing that, as expected, the reported measures do not generally consider those samples to be OOD. We now look at the AUROCs obtained with S (Table 1). We see that, not only are reversed for less complex datasets like MNIST or SVHN, but also that all AUROCs for the rest of the data sets improve as well. The only exception to the last assertion among all studied combinations is the combination of TinyImageNet with PixelCNN++ and FLIF (see Appendix D for other combinations). In general, we obtain AUROCs above 0.7, with many of them approaching 0.9 or 1. Thus, we can conclude that S clearly improves over likelihoods alone in the OOD detection task, and that S is able to revert the situation with intuitively less complex data sets that were previously yielding a low AUROC. We also study how the training set, the choice of compressor/generative model, or the size of the model affects the performance of S (Appendix D). In terms of models and compressors, we do not observe a large difference between the considered combinations, except for a few isolated cases whose investigation we defer for future work. In terms of model size, we do observe a tendency to provide better discrimination with increasing size. In terms of data sets, we find the OOD detection task to be easier with FashionMNIST than with CIFAR10. We assume that this is due to the ease of the generative model to learn and approximate the density conveyed by the data. A similar but less marked trend is also observed for compressors, with better compressors yielding slightly improved AUROCs than other, in principle, less powerful ones. A takeaway from all these observations would be that using larger generative models and better compressors will yield a more reliable S and a better AUROC. The conducted experiments support that, but a more in-depth analysis should be carried out to further confirm this hypothesis. Finally, we want to assess how S compares to previous approaches in the literature. For that, we compile a number of reported AUROCs for both classifier-and generative-based approaches and compare them with S. Note that classifier-based approaches, as mentioned in Sec. 1, are less applicable than generative-based ones. In addition, as they exploit label information, they might have an advantage over generative-based approaches in terms of performance (some also exploit external or outlier data; Sec. 4). We observe that S is competitive with both classifier-and existing generative-based approaches (Table 2). When training with FashionMNIST, S achieves the best scores among all considered approaches. The with further test sets are also encouraging, with almost all AUROCs approaching 1 (Appendix D). When training with CIFAR10, S achieves similar or better performance than existing approaches. Noticeably, within generative-based approaches, S is only outperformed in two occasions by the same approach, WAIC, which uses ensembles of generative models (Sec. 4). On the one hand, it would be interesting to see how S could perform when using ensembles of models and compressors to produce better estimates of − M and L, respectively. On the other hand, however, the use of a single generative model together with a single fast compression library makes S an efficient alternative compared to WAIC and some other existing approaches. It is also worth noting that many existing approaches have a number of hyper-parameters that need to be tuned, sometimes with the help of outlier or additional data. In contrast, S is a parameter-free measure, which makes it easy to use and deploy. We illustrate a fundamental insight with regard to the use of generative models' likelihoods for the task of detecting OOD data. We show that input complexity has a strong effect in those likelihoods, and pose that it is the main culprit for the puzzling of using generative models' likelihoods for OOD detection. In addition, we show that an estimate of input complexity can be used to com- , (b) by , and (c) by. Results for Typicality test correspond to using batches of 2 samples of the same type. Trained on: FashionMNIST CIFAR10 OOD data: MNIST Omniglot SVHN CelebA CIFAR100 Classifier-based approaches ODIN 0.766 0.796 1.000 0.997 -Outlier exposure --0.758 -0.685 Typicality test (b) 0.140 -0.420 --Likelihood-ratio 0.997 -0.912 --S using Glow and FLIF (ours) 0.998 1.000 0.950 0.863 0.736 S using PixelCNN++ and FLIF (ours) 0.967 1.000 0.929 0.776 0.535 pensate standard negative log-likelihoods in order to produce an efficient and reliable OOD score. We also offer an interpretation of our score as a likelihood-ratio akin to Bayesian model comparison. Such score performs comparably to, or even better than several state-of-the-art approaches, with that are consistent across a range of data sets, models, model sizes, and compression algorithms. The proposed score has no hyper-parameters besides the definition of a generative model and a compression algorithm, which makes it easy to employ in a variety of practical problems and situations. In our experiments, we employ well-known, publicly-available data sets. In addition to those, and to facilitate a better understanding of the problem, we develop another two self-created sets of synthetic images: Noise and Constant images. The Noise data set is created by uniformly randomly sampling a tensor of 3×32×32 and quantizing the to 8 bits. The Constant data set is created similarly, but using a tensor of 3×1×1 and repeating the values along the last two dimensions to obtain a size of 3×32×32. The complete list of data sets is available in Table 3. In the case of data sets with different variations, such as CelebA or FaceScrub, which have both plain and aligned versions of the faces, we select the aligned versions. Note that, for models trained on CIFAR10, it is important to notice the overlap of certain classes between that and other sets, namely TinyImageNet and CIFAR100 (they overlap, for instance, in classes of certain animals or vehicles). Therefore, strictly speaking, such data sets are not entirely OOD, at least semantically. In order to split the data between train, validation, and test, we follow two simple rules: if the data set contains some predefined train and test splits, we respect them and create a validation split using a random 10% of the training data; if no predefined splits are available, we create them by randomly assigning 80% of the data to the train split and 10% to both validation and test splits. In order to create consistent input sizes for the generative models, we work with 3-channel images of size 32×32. For those data sets which do not match this configuration, we follow a classic bi-linear resizing strategy and, to simulate the three color components from a gray-scale image, we triplicate the channel dimension. The of this paper are obtained using two generative models of different nature: one autoregressive model and one invertible model. As autoregressive model we choose PixelCNN++ , which has been shown to obtain very good in terms of likelihood for image data. As invertible model we choose Glow , which is also capable of inferring exact log-likelihoods using large stacks of bijective transformations. We implement the Glow model using the default configuration of the original implementation 1, except that we zeropad and do not use ActNorm inside the coupling network. The model has 3 blocks of 32 flows, using an affine coupling with an squeezing factor of 2. As for PixelCNN++, we set 5 residual blocks per stage, with 80 filters and 10 logistic components in the mixture. The non-linearity of the residual layers corresponds to an exponential linear unit 2. We train both Glow and PixelCNN++ using the Adam optimizer with an initial learning rate of 10 −4. We reduce this initial value by a factor of 1/5 every time that the validation loss does not decrease during 5 consecutive epochs. The training finishes when the learning rate is reduced by factor of 1/100. The batch size of both models is set to 50. The final model weights are the ones yielding the best validation loss. The likelihoods obtained in validation with both Glow and PixelCNN++ match the ones reported in the literature for CIFAR10 . We also make sure that the generated images are of comparable quality to the ones shown in those references. We use PyTorch version 1.2.0 . All models have been trained with a single NVIDIA GeForce GTX 1080Ti GPU. Training takes some hours under that setting. We explore three different options to compress input images. As a mandatory condition, they need to provide lossless compression. The first format that we consider is PNG, and old-classic format which is globally used and well-known. We use OpenCV 3 to compress from raw Numpy matrices, with compression set to the maximum possible level. The second format that we consider is JPEG2000. Although not as globally known as the previous one, it is a more modern format with several new generation features such as progressive decoding. Again, we use the default OpenCV implementation to obtain the size of an image using this compression algorithm. The third format that we consider is FLIF, the most modern algorithm of the list. According to its website 4, it promises to generate up to 53% smaller files than JPEG2000. We use the publicly-available compressor implementation in their website. We do not include header sizes in the measurement of the ing bits per dimension. To compute our complexity estimate L(x), we compress the input x with one of the compressors C above. With that, we obtain a string of bits C(x). The length of it, |C(x)|, is normalized by the size or dimensionality of x, which we denote by d, to obtain the complexity estimate: We also experimented with an improved version of L, where L i corresponds to different compression schemes. This forces S to work always with the best compressor for every x. In our case, as FLIF was almost always the best compressor, we did not observe a clear difference between using L or L. However, in cases where it is not clear which compressor to use or cases in which we do not have a clear best/winner, L could be of use. The additional mentioned in the main paper are the following: • In Table 4, we report the average log-likelihood M for every data set. We sort data sets from highest to lowest log-likelihood. • In Table 5, we report the global Pearson's correlation coefficient for different models, train sets, and compressors. Due to the large sample size, Scipy version 1.2.1 reports a p-value of 0 in all cases. • In Table 6, we report the AUROC values obtained from log-likelihoods M, complexity estimates L, a simple two-tail test T taking into account lower and higher log-likelihoods, T = | M − M |, and the proposed score S. • In Table 7, we report the AUROC values obtained from S across different Glow model sizes, using a PNG compressor. • In Table 8, we report the AUROC values obtained from S across different data sets, models, and compressors. Table 6: AUROC values using negative log-likelihood − M, the complexity measure L, a simple two-tail test T (see text), and our score S for Glow and PixelCNN++ models trained on CIFAR10 and using a PNG compressor. Qualitatively similar were obtained for FashionMNIST and other compressors..000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

Represent the TLDR for finding the corresponding scientific article

We pose that generative models' likelihoods are excessively influenced by the input's complexity, and propose a way to compensate it when detecting out-of-distribution inputs

represent the tldr for finding the corresponding scientific article:

E.g.:
A novel domain adaptation method to align manifolds from source and target domains using label propagation for better accuracy. == The difficulty of obtaining sufficient labeled data for supervised learning has motivated domain adaptation, in which a classifier is trained in one domain, source domain, but operates in another, target domain. Reducing domain discrepancy has improved the performance, but it is hampered by the embedded features that do not form clearly separable and aligned clusters. We address this issue by propagating labels using a manifold structure, and by enforcing cycle consistency to align the clusters of features in each domain more closely. Specifically, we prove that cycle consistency leads the embedded features distant from all but one clusters if the source domain is ideally clustered. We additionally utilize more information from approximated local manifold and pursue local manifold consistency for more improvement. Results for various domain adaptation scenarios show tighter clustering and an improvement in classification accuracy. Classifiers trained through supervised learning have many applications , but it requires a great deal of labeled data, which may be impractical or too costly to collect. Domain adaptation circumvents this problem by exploiting the labeled data available in a closely related domain. We call the domain where the classifier will be used at, the target domain, and assume that it only contains unlabeled data {x t}; and we call the closely related domain the source domain and assume that it contains a significant amount of labeled data {x s, y s}. Domain adaptation requires the source domain data to share discriminative features with the target data . In spite of the common features, a classifier trained using only the source data is unlikely to give satisfactory in the target domain because of the difference between two domains' data distributions, called domain shift . This may be addressed by fine-tuning on the target domain with a small set of labeled target data, but it tends to overfit to the small labeled dataset . Another approach is to find discriminative features which are invariant between two domains by reducing the distance between the feature distributions. For example, domain-adversarial neural network (DANN) achieved remarkable using generative adversarial networks (GANs) . However, this approach still has room to be improved. Because the classifier is trained using labels from the source domain, the source features become clustered, and they determine the decision boundary. It would be better if the embedded features from the target domain formed similar clusters to the source features in class-level so that the decision boundary does not cross the target features. Methods which only reduce the distance between two marginal distributions bring the features into general alignment, but clusters do not match satisfactorily, as shown in Fig. 1(a). As a consequence, the decision boundary is likely to cross the target features, impairing accuracy. In this work, we propose a novel domain adaptation method to align the manifolds of the source and the target features in class-level, as shown in Fig. 1(b). We first employ label propagation to evaluate the relation between manifolds. Then, to align them, we reinforce the cycle consistency that is the correspondence between the original labels in the source domain and the labels that are propagated from the source to the target and back to the source domain. The cycle consistency draws features from both domains that are near to each other to converge, and those that are far apart to diverge. The proposed method exploits manifold information using label propagation which had not been taken into account in other cycle consistency based methods. As a , our approach outperforms other baselines on various scenarios as demonstrated in Sec. 4. Moreover, the role of cycle consistency is theoretically explained in Sec. 3.2 that it leads to aligned manifolds in class-level. To acquire more manifold information within the limited number of mini-batch samples, we utilize local manifold approximation and pursue local manifold consistency. In summary, our contributions are as follows: • We propose a novel domain adaptation method which exploits global and local manifold information to align class-level distributions of the source and the target. • We analyze and demonstrate the benefit of the proposed method over the most similar baseline, Associative domain adaptation (AssocDA) . • We present the theoretical on why the proposed cycle consistency leads to class-level manifold alignment, bringing better in domain adaptation. • We conduct extensive experiments on various scenarios and achieve the state-of-the-art performance. Unsupervised Domain Adaptation It has been shown that the classification error in the target domain is bounded by that in the source domain, the discrepancy between the domains and the difference in labeling functions. Based on this analysis, a number of works have endeavored to train domain-confusing features to minimize the discrepancy between the domains (; ; ; 2017). Maximum mean discrepancy can be used as a measure of domain discrepancy. In an approach inspired by GANs, a domain confusion can be converted into a minmax optimization. While minimization of domain discrepancy can be effective in reducing the upper bound on the error, it does not guarantee that the feature representation in the target domain is sufficiently discriminative. To address this issue, several techniques had been proposed. Explicit separation of the shared representation from the individual characteristics of each domain may enhance the accuracy of the model . This approach has been implemented as a network with private and shared encoders and a shared decoder. The centroid and prototype of each category can be used for class-level alignment . An alternative to such featurespace adaptation techniques is the direct conversion of target data to source data (; ;). Those proposed methods intend to transfer the style of images to another domain while preserving the content. This performs well on datasets containing Figure 2: Overview of our method. The feature generator G projects the input data into the feature space. The dashed line means weight sharing. The embedded source features f s and the target features f t are organized into a graph and then used together to evaluate cycle consistency through label propagation. The embedding classifier C learns from the source ground-truth labels. The discriminator D determines whether features originated in the source or the target domain. images that are similar at the pixel-level; they are problematic when the mapping between high-level features and images is complicated . Metric Learning Metric learning is learning an appropriate metric distance to measure the similarity or dissimilarity between data . Reducing the distances between similar data and increasing the distances between distinct data has shown to improve the accuracy of a classifier. Metric learning is particularly beneficial when very little labeled data is available, which is the situation for domain adaptation. combined metric learning and unsupervised domain adaptation with the enforcement of cycle consistency. In particular, the inner products of source features and target features with the same label are maximized, and minimized between features with different labels. AssocDA enforces the feature alignment between the source and target by forcing the two step round trip probability to be uniform in the same class and to vanish between different classes. Graph-based learning is closely related to metric learning, in that it achieves clustering using distance information. Label consistency is usually assumed, meaning that adjacent data tend to have the same labels . Label propagation has improved the performance of semi-supervised learning by enforcing label consistency by propagating labels from labeled to unlabeled data. To overcome need for fixed graphs to be provided in advance, the distances between each node can be adaptively learned , as in metric learning, and this increases accuracy in both semi-supervised and few-shot learning. Our algorithm, shown in Fig. 2, uses label propagation and cycle consistency to learn features from the source and the target domains which are both 1) indistinguishable each other and 2) close when placed within the same class, but distant when placed in different classes. The details are as follows. Manifold learning extracts intrinsic structures from both unlabeled and labeled data. We obtain these structures by constructing a graph whose vertexes are the embedded features and whose edges are the relations between data. We first embed the input data in the feature space, using the feature generator composed of convolutional layers following previous work . Subsequently, a fully connected graph is constructed according to the distances between the features. The edge weights W ij between the input data x i, x j are determined from the feature vectors using Gaussian similarity, ), where f i, f j are the embedded feature vectors of x i, x j, and σ is a scale parameter. It is known that graph-based methods are sensitive to the scale parameter σ. A large σ in an uniformly connected graph that disregards the latent structure of the data, while a small σ produces a sparse graph which fails to express all the relationship between the data. To adapt σ according to the embedded features, we take σ as a trainable variable to be learned during training. Label propagation is a method of manifold regularization, which in turn produces a classifier that is robust against small perturbations. Label propagation can be seen as a repeated random walk through the graph of features using an affinity matrix to assign the labels of target data . A label matrix y n ∈ R (Ns+Nt)×C refers to the labels assigned to data in both domains at the n-th step random walk. The dimension of y n is determined by N s, N t, and C which are the numbers of source and target data points and the number of classes, respectively. The first N s rows of y n contain the labels of the source data, and the remaining N t rows contain the labels of the target data. The initial label vector y 0 contains y s for the source data, which is one-hot coded ground-truth labels and zero vectors for the target data. The one step of the random walk transforms the label vector as follows: where,. W ts is a similarity matrix between the target and source data, and W tt is a similarity matrix which represents the interrelations in the target data. These are described in the Sec. 3.1. The normalization operation normalize(·) transforms the sum of each row to 1. The identity matrix in the normalized transition matrix T signifies that the labels of source data do not change because its labels are already known. In graph theory, these source data points would be called absorbing nodes. In label propagation, the labels of the target domain is assigned to the propagated labelsŷ t by infinite transition, formulated asŷ tt T ts y s, which converges as follows In our method,ŷ t is used to obtain the propagated labels of the source data in the same way aŝ y s = (I − T ss) −1 T stŷ t where T ss and T st are defined analogous to T tt and T ts, so that we can learn the features of which clusters match each other. We then refer to the property thatŷ s should be the same as the original label y s as cycle consistency. Pursuing cycle consistency forces not perfectly aligned features to move toward the nearest cluster, as shown in Fig. 3. The following theorem shows that enforcing cycle consistency on ideally clustered source data will segregate different classes of the source and the target data and gather the same classes. Theorem 1. Let {e i |1 ≤ i ≤ C} be the standard bases of C-dimensional Euclidean space. For the sake of simplicity, source data x 1, x 2, · · ·, x Ns are assumed to be arranged so that the first n 1 data belong to class 1, the n 2 data to class 2, and so forth. Assume that 1) the source data is ideally clustered, in the sense that T ss has positive values if the row and the column are the same class and zero otherwise, i.e., T ss = diag(T 1, T 2, · · ·, T C), the block diagonal where T i is a n i × n i positive matrix for i = 1, 2, · · ·, C and 2)ŷ s = y s. Then for all 1 ≤ j ≤ C, there exists a nonnegative vector v j ∈ R Ns such that 1) the part where source data belongs to j th class (from th element to [n 1 + n 2 + · · · + n j] th element) are positive and the other elements are all zero and 2) v j T stŷ t e i = 0 for all 1 ≤ i ≤ C, i = j. Proof. The illustration and the proof is given in Appx. A. In Thm. 1,ŷ t e i refers to the assigned probability as i th class to the target data. The implies that if a target data is enough to be predicted as i th class through label propagation, i.e., i th elements of the row inŷ t corresponding to the target data is nonzero, then the elements of T st which represent the transitions from source data of all but i th class to the target data should vanish, i.e., the target data is segregated from the source data in different classes. As described in Sec. 3.4, we employed DANN to prevent the target data distribution to be distinct from the source data distribution. If a column of T st is a zero vector, the feature of the corresponding target data for the column is considerably distant from all source data features. However, minimizing the DANN loss makes target features lie around source features, and thus each column of T st is not likely to be a zero vector. Combining this conjecture with Thm. 1, each row ofŷ t has only one nonzero value, i.e., every target data belongs to only one cluster. We thus argue that by pursuing this property, generator can learn more discriminative shared features, and classification performance may improve. Cycle consistency is enforced by minimizing the l 1 loss L cycle betweenŷ s and y s: Comparison with AssocDA The proposed method has some resemblance with AssocDA in that they both consider the similarities and transitions between data. However, we argue that AssocDA is a special case of our method. First, our method exploits manifold over each domain by taking relations within the same domain into account through label propagation, whereas AssocDA only considers relations across the domains. Specifically, in Eq. 1, our method utilizes both T ts and T tt, but AssocDA ignores T tt which often has useful information about the target data manifold. Second, AssocDA forces the two-step transition to be uniform within the same class. This strict condition may drive the source features of each class to collapse to one mode and can cause overfitting. On the contrary, our method only constrains source data to preserve its original labels after the label propagation. Thus, it does not require all source data be close to each other within the same class; it allows moderate intra-class variance. The experiment in Sec. 4.1 and Fig. 4 support these arguments and visualize the effect of the differences. As shown in Thm. 1, the introduced cycle consistency utilizes graph based global manifold information and enforces the source and target features to be aligned in class-level. However, in practice, the limited size of mini-batch may restrict the available information of graph. The knowledge from the local manifold of each sample, in this case, can complement the global manifold information. In this regard, we additionally pursue local manifold consistency that the output should not be sensitive to small perturbations in the local manifold, as suggested elsewhere (; ;). Concretely, localized GAN (LGAN) is employed to approximate the local manifold of each data and sample a marginally perturbed image along the local manifold from the given data. LGAN allows it as LGAN focuses on learning and linking patches of local manifolds in its training procedure. The difference between the predicted label of the perturbed image and that of the original image is minimized to impose local manifold consistency of the classifier as follows: where, C, G and G L are the embedding classifier, the feature generator and the LGAN generator, respectively. LGAN generator, G L (x, z), takes an image x and noise z to generate locally perturbed image along the approximated local manifold. H(·, ·) denotes cross entropy. µ and η are coefficients for the source and the target local manifold consistency loss, respectively. Our method learns a clustered feature representation that is indistinguishable across the source and target domains through the training process as follows: where, D is the discriminator. α and β are coefficients for the last two terms and λ is a scheduling parameter described in Appx B.1. L class is a widely used cross-entropy loss for labeled source data and L dann is a GAN loss : where discriminator's output D(·) is the probability that the input originated from the source domain. From the metric learning perspective, L class serves to separate the source features according to their ground-truth labels, which supports the assumption in Thm. 1, the ideally clustered source features. Subsequently, L dann takes a role in moving the target features toward the source features, but it is insufficient to produce perfectly aligned clusters. Our cycle loss L cycle and local loss L local facilitate clustering by enforcing cycle consistency and local manifold consistency. We present a toy example to empirically demonstrate the effect of our proposed cycle loss using manifold information compared to the most similar method, AssocDA. We designed synthetic dataset in 2-dimensional feature space with two classes as illustrated in the leftmost of Fig. 4. The source data lie vertically and the target data are slightly tilted and translated. The second column shows the negative gradients of AssocDA loss and our cycle loss with respect to each data. Negative gradients can be interpreted as the movement of features at each iteration. The third and fourth are the updated features using gradient descent in the middle and at the end of feature updates 1. As argued in Sec. 3.2, AssocDA does not consider the transition within the same domain and thus target data which are close to source data with different label (points inside red circles in the second column) are strongly attracted to them. On the other hand, the gradients of the cycle loss are much smaller than AssocDA. We speculate that it is because the attractions from source data in the same class are propagated through target data manifold. As a , AssocDA leads some data to move in wrong direction, being misclassified, while cycle loss brought correctly aligned manifolds. In addition, AssocDA attracts all features too close at the end of updates, which may cause overfitting. Last but not least, our cycle loss aligned source and target clusters correctly without the aid of dann loss. We thus argue that our method is complementary to DANN rather than an extension. We show the performance of the proposed method on two real visual dataset. First dataset, which we call by Digit & Object dataset, includes digit dataset such as SVHN and Synthetic Digits (DIGITS), Gradients of loss Progress at 150 steps Progress at 600 steps Figure 5: Visualization of learned features using t-SNE. Circles and x markers respectively indicate the source and target features. Colors correspond to labels. In all cases, the features from two domains form similar and tight clusters, which is the key objective of our method. and object dataset such as STL and CIFAR. We used ImageCLEF-DA as second dataset for more challenging benchmark. We employed three networks as previous work (; ; . A network with two convolutional layers and two fully connected layers for digit dataset and a network with nine convolutional layers and one fully connected layer for object dataset were implemented. Pretrained ResNet was used for ImageCLEF-DA dataset. More details on training settings, adaptation scenarios and an experiment on non-visual dataset are provided in Appx. B.1, B.2 and E. Tab. 1 compares the accuracy of our method on Digit & Object dataset with that of other approaches. For our method, we reported the of three models, one with local loss (L), another with cycle loss (C) and the other with both losses (C+L). Our algorithm outperformed the others on most of the tasks. In the most experiments, the performance of the proposed method was better than the state-ofthe-art. This suggests that enforcing alignment in addition to domain-invariant embedding reduces the error-rate. PixelDA showed superior performance on MNIST→MNIST-M, but it is attributable to the fact that PixelDA learns transferring the style of images at a pixel level which is similar to the way MNIST-M is generated from MNIST. T-SNE embeddings in Fig. 5 indicates that the learned features are well aligned and clustered. Tab. 2 reports the on ImageCLEF-DA dataset experiments. The performance of our method was better than or comparable to those of other baselines. Especially, our method outperforms which also aims to learn clustered and aligned features. Although the objectives are related, the approaches are quite different. Our method utilizes the manifolds of the source and the target domain through label propagation and cycle consistency, whereas CAT considers the distance between two samples for clustering and the distance between the first-order statistics of distributions for alignment. We argue that the better performance is attributed to utilizing manifold information beyond one to one relations of which benefits are explained in Sec. 4.1. Throughout ImageCLEF-DA experiments, the proposed method without the local loss achieved better accuracy compared to that with the local loss. Approximation of the local manifold on ImageCLEF-DA generated by LGAN was slightly worse than that on Digit & Object dataset; perturbed image was blurred and semantically invariant with the original image. Hence, we speculate that the performance of the proposed method may be improved with better local manifold approximation. In this paper, we proposed a novel domain adaptation which stems from the objective to correctly align manifolds which might in better performance. Our method achieved it, which was supported by intuition, theory and experiments. In addition, its superior performance was demonstrated on various benchmark dataset. Based on graph, our method depends on how to construct the graph. Pruning the graph or defining a similarity matrix considering underlying geometry may improve the performance. Our method also can be applied to semi supervised learning only with slight modification. We leave them as future work. A PROOF OF THEOREM 1 Theorem 1. Let {e i |1 ≤ i ≤ C} be the standard bases of C-dimensional Euclidean space. For the sake of simplicity, source data x 1, x 2, · · ·, x Ns are assumed to be arranged so that the first n 1 data belong to class 1, the n 2 data to class 2, and so forth. Assume that 1) the source data is ideally clustered, in the sense that T ss has positive values if the row and the column are the same class and zero otherwise, i.e., T ss = diag(T 1, T 2, · · ·, T C), the block diagonal where T i is a n i × n i positive matrix for i = 1, 2, · · ·, C and 2)ŷ s = y s. Then for all 1 ≤ j ≤ C, there exists a nonnegative vector v j ∈ R Ns such that 1) the part where source data belongs to j th class (from [n 1 + n 2 + · · · + n j−1 + 1] th element to [n 1 + n 2 + · · · + n j] th element) are positive and the other elements are all zero and 2) v j T stŷ t e i = 0 for all 1 ≤ i ≤ C, i = j. From the assumption, T ss is a block diagonal matrix of which block elements are T 1,T 2,· · ·,T C. v j is all zero except n j elements in the middle of v j. The n j elements are all positive and their indices correspond to those of T j in T ss. In the proof, the left eigenvector u j of T j will be substituted to this part. Proof. From the Perron-Frobenius Theorem that positive matrix has a real and positive eigenvalue with positive left and right eigenvectors, T j, the block diagonal element of T ss, has a positive left eigenvector u j with eigenvalue λ j for all j = 1, 2, · · · C. Then, as shown below, v j = (0 0 ··· 0 u j 0 ··· 0) where n 1 + n 2 + · · · + n j−1 zeros, u j and n j+1 + n j+2 + · · · + n C zeros are concatenated, is a left eigenvector of T ss with eigenvalue λ j by the definition of eigenvector. From the label propagation, we have,ŷ By multiplying v j (I − T ss) on the left and e i on the right to the both sides in Equation 13 and combining with the assumptionŷ s = y s, we have, The last zero comes from the definition of v j. In this subsection, we offer the modified version of Thm. 1 when the source features are slightly perturbed from the ideally clustered condition and the other assumption y s =ŷ s holds. We start from representing T ss as follows to indicate the perturbation. where, δT ss is assumed to be sufficiently small under infinite norm and T ss is a block diagonal transition matrix when the source features are ideally clustered as stated in Thm. 1. In the proof above, we showed eigenvalue λ j and its corresponding eigenvector v j of T j. According to perturbation theory of eigenvalue and eigenvector , the eigenvector can be approximated by first order when the perturbation is small. More generally and precisely, where, the norm is vector or matrix 2-norm and m j is determined by T ss. For the sake of simplicity, we use Big-O notation in Eq. 19 and Eq. 20. Now, we reuse Eq. 16 from the proof of Theorem 1 since it is still valid under the modified condition. We apply Eq. 19 to the right hand side as follows, where i = j. Eq. 24 holds because only j th block elements of v j are nonzero. We also used the fact that y s is bounded by 0 and 1. Similarly, the left hand side of Eq. 22 can be transformed as follows, The second term of Eq. 26 holds because T st andŷ t are bounded by 0 and 1. Finally, by combining Eq. 24 and Eq. 26, we have, Eq. 27 implies that if the perturbation is sufficiently small i.e., ||δT ss || ∞ << 1 and a target data is enough to be predicted as i th class through label propagation, then the transitions from source data of all but i th class to the target data is negligible because v j is positive for j th block and zero for others. It is the same with the of Theorem 1. In addition, the more strongly the target data is classified as i th class i.e., the corresponding element ofŷ t becomes greater, the smaller the transitions from source data in the other classes are, indicating the segregation against the other classes. Practically, the coefficients for L cycle and L cycle are scheduled to facilitate the clustering of source features correctly at the early stage of training. Thus we may assume that T ss is marginally perturbed around the ideally clustered one when our cycle loss takes effect. Scheduling the effect of losses To reduce the effect of noisy signal from L dann and L cycle during the early stages of training, a weight balance factor λ = 2 1+exp(−γ·p) − 1 is applied in Eq. 6. A constant γ determines the rate of increase of λ; p is the progress of training, which proceeds from 0 to 1. The parameter was introduced to make a classifier less sensitive to the erroneous signals from the discriminator in the beginning. Throughout the experiments, γ was set to 10. Hyperparameter Although it would be ideal to avoid utilizing labels from the target domain in the hyperparameter optimization, it seems that no globally acceptable method exists for this. One possibility is reverse validation scheme but this may not be accurate enough to estimate test accuracy . In addition , applications exist where the labeled target domain data is available at the test phase but not at the training phase. Hence, we adopted the protocol of that exploits a small set of labeled target domain data as a validation set; 256 samples for the Amazon review experiment and 1,000 samples for the other experiments (; 2017;). During training, Adam optimizer with learning rate of 10 −3 was utilized. Exponential moving averaging was applied to the optimization trajectory. It is an inherent characteristic of our method that each data sample affects the graph structure. So it is important for each class sample in each batch to represent its classes accurately. In other words, the transition matrix can be corrupted by biases in the samples. Therefore, the number of data samples in each class in a batch should be sufficient to avoid any likely bias. To address this problem, we performed experiments with batch size of up to 384 and observed very little improvement beyond a batch size of 128. So we fixed the batch size to 128 for Digit & Object dataset. For the ImageCLEF-DA dataset, we set the batch size to 36 because of limited computing resource. MNIST → MNIST-M The MNIST database of hand-written digits consists of digit images with 10 classes and MNIST-M consists of MNIST digits blended with natural color patches from the BSDS500 dataset . In addition, following other work the colors of the MNIST images were inverted randomly, because their colors are always white on black, whereas the MNIST-M images exhibit various colors. MNIST ↔ USPS USPS is another dataset of hand-written images of digits, with 10 classes. USPS contains 16×16 images and the size of the USPS image is upscaled to 28×28, which is the size of the MNIST image in our experiment. The evaluation protocol of CYCADA is adopted. SVHN → MNIST The Street View House Numbers (SVHN) dataset consists of images of house numbers acquired by Google Street View. The natural images that it contains, are substantially different from the line drawings in the MNIST dataset. The size of each MNIST image is upscaled to 32×32, which is the size of SVHN images. SYN DIGITS → SVHN SYN DIGITS dataset is synthetic number dataset which is similar to the SVHN dataset . The most significant difference between the SYN DIGITS dataset and the SVHN dataset is the untidiness in the of real images. CIFAR ↔ STL Both CIFAR dataset and STL dataset are 10-class datasets that contain images of animals and vehicles. Not overlapped classes are removed to make a 9-class domain adaptation task . We used the larger network only for this experiment. 2 The twelve common classes of three publicly available dataset (Caltech-256, ImageNet ILSVRC2012, and PASCAL VOC2012) are selected to form visual domain adaptation tasks. We perform all six possible adaptation scenarios among these three dataset. We searched hyperparameters within α = {0, 0.01, 0.1, 1}, β = {0.01, 0.1, 1}, µ = {0, 0.01} and η = {0, 0.1}. Perturbation to the LGAN generator, i.e. z, is fixed to 0.5 for all experiments. The best hyperparameters for each task is shown in Table. 3. Setting an appropriate value for the scale parameter, σ, is important because it has a substantial role in determining the transition matrix, T. Therefore, we conducted several experiments with fixing σ to various values. For these experiments, we excluded L local to observe the effect of σ.' Adapt' means that the σ is learned to adapt according to the embedded features. For four out of five scenarios, fixing σ to 1 performed better than fixing it to 0.1 or 10. With this observation, we initialized σ to 1 took it as a trainable variable. The of adaptively learning σ is reported at the bottom row of the table. Compared to fixing σ to 1, adaptively learning σ achieved better accuracy and had a lower standard deviation range which means that it is more stable. We also would like to highlight that our model is robust to the initial value of σ. We conducted extensive experiments with initializing σ to 0.1, 1 and 10 and taking it as a trainable variable. Except for SVHN → MNIST transfer task with setting initial σ value to 10, the initial value of σ has a minute influence to the accuracy. We believe that adaptively learning the scale parameter can be usefully employed in any other graph-based method. The learned σ values for various scenarios are as follows. It seems that σ adaptively learns its value according to the transfer task, regardless of its initialization. We tried l 2 loss and cross entropy loss to enforce cycle consistency as well. We excluded L local to compare the effectiveness of these functions. For all Digit dataset adaptation experiments, evaluating cycle consistency with l 1 norm achieved the highest accuracy. We speculate that l 1 norm is more numerically stable or provides more effective gradients than other functions in this case. The Amazon Reviews dataset provides a non-visual domain for domain adaptation experiments. It contains reviews of books, DVDs, electronics, and kitchen appliances encoded as 5,000-dimensional feature vectors containing unigrams and bigrams of the texts with binary labels. Four-and five-star reviews are labeled'positive'; reviews with fewer stars are labeled'negative'. We used 2,000 labeled source data and 2,000 unlabeled target data for training, and between 3,000 to 6,000 target data for testing. Tab. 8 shows that our method performs better than DANN , VFAE and ATT on the Amazon Reviews data in six out of twelve experiments. Our method was more accurate than DANN in nine out of twelve settings, showing approximately 2.0% higher classification accuracy on average.

This work aims to provide quantitative answers to the relative importance of concepts of interest via concept activation vectors (CAV). In particular, this framework enables non-machine learning experts to express concepts of interest and test hypotheses using examples (e.g., a set of pictures that illustrate  the concept). We show that CAV can be learned given a relatively small set of examples. Hypothesis testing with CAV can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other sets of concepts. Interpreting networks with CAV does not require any retraining or modification of the network.