Question: Represent the article for finding a single-sentence TLDR

Question: An important question in task transfer learning is to determine task transferability, i.e. given a common input domain, estimating to what extent representations learned from a source task can help in learning a target task. Typically, transferability is either measured experimentally or inferred through task relatedness, which is often defined without a clear operational meaning. In this paper, we present a novel metric, H-score, an easily-computable evaluation function that estimates the performance of transferred representations from one task to another in classification problems. Inspired by a principled information theoretic approach, H-score has a direct connection to the asymptotic error probability of the decision function based on the transferred feature. This formulation of transferability can further be used to select a suitable set of source tasks in task transfer learning problems or to devise efficient transfer learning policies. Experiments using both synthetic and real image data show that not only our formulation of transferability is meaningful in practice, but also it can generalize to inference problems beyond classification, such as recognition tasks for 3D indoor-scene understanding. Transfer learning is a learning paradigm that exploits relatedness between different learning tasks in order to gain certain benefits, e.g. reducing the demand for supervision BID22 ). In task transfer learning, we assume that the input domain of the different tasks are the same. Then for a target task T T, instead of learning a model from scratch, we can initialize the parameters from a previously trained model for some related source task T S. For example, deep convolutional neural networks trained for the ImageNet classification task have been used as the source network in transfer learning for target tasks with fewer labeled data BID7 ), such as medical image analysis BID24 ) and structural damage recognition in buildings (Gao & Mosalam). An imperative question in task transfer learning is transferability, i.e. when a transfer may work and to what extent. Given a metric capable of efficiently and accurately measuring transferability across arbitrary tasks, the problem of task transfer learning, to a large extent, is simplified to search procedures over potential transfer sources and targets as quantified by the metric. Traditionally, transferability is measured purely empirically using model loss or accuracy on the validation set (; ; BID5). There have been theoretical studies that focus on task relatedness BID1; BID19; BID21; BID2 ). However, they either cannot be computed explicitly from data or do not directly explain task transfer performance. In this study, we aim to estimate transferability analytically, directly from the training data. We quantify the transferability of feature representations across tasks via an approach grounded in statistics and information theory. The key idea of our method is to show that the error probability of using a feature of the input data to solve a learning task can be characterized by a linear projection of this feature between the input and output domains. Hence we adopt the projection length as a metric of the feature's effectiveness for the given task, and refer to it as the H-score of the feature. More generally, H-score can be applied to evaluate the performance of features in different tasks, and is particularly useful to quantify feature transferability among tasks. Using this idea, we define task transferability as the normalized H-score of the optimal source feature with respect to the target task. As we demonstrate in this paper, the advantage of our transferability metric is threefold. (i) it has a strong operational meaning rooted in statistics and information theory; (ii) it can be computed directly and efficiently from the input data, with fewer samples than those needed for empirical learning; (iii) it can be shown to be strongly consistent with empirical transferability measurements. In this paper, we will first present the theoretical of the proposed transferability metric in Section 2-4. Section 5 presents several experiments on real image data, including image classificaton tasks using the Cifar 100 dataset and 3D indoor scene understanding tasks using the Taskonomy dataset created by. A brief review of the related works is included in Section 6. In this section, we will introduce the notations used throughout this paper, as well as some related concepts in Euclidean information theory and statistics. X, x, X and P X represent a random variable, a value, the alphabet and the probability distribution respectively. √ P X denotes the vector with entries P X (x) and [√ P X] the diagonal matrix of P X (X). For joint distribution P Y X, P Y X represents the |Y| × |X | probability matrix. Depending on the context, f(X) is either a |X |-dimensional vector whose entries are f (x), or a |X | × k feature matrix. Further, we define a task to be a tuple T = (X, Y, P XY), where X is the training features and Y is the training label, and P XY the joint probability (possibly unknown). Subscripts S and T are used to distinguish the source task from the target task. Our definiton of transferability uses concepts in local information geometry developed by BID18, which characterizes probability distributions as vectors in the information space. Consider the following binary hypothesis testing problem: test whether i.i.d. samples DISPLAYFORM0 are drawn from distribution P 1 or distribution P 2, where P 1, P 2 belong to an -neighborhood N (P 0) {P | x∈X DISPLAYFORM1 as the information vector corresponding to P i for i = 1, 2. DISPLAYFORM2 for the binary hypothesis testing problem. Let E f be the error exponent of decision region {x m | l(x m) > T } for T ≥ 0, which characterizes the asymptotic error probability P e of l (i.e. lim m→∞ − 1 m log(P e) = E k f ). E f can be written as the squared length of a projection: DISPLAYFORM3 When f (x) = log DISPLAYFORM4 P2(x) is the log likelihood ratio, l is the minimum sufficient statistics that achieves the largest error exponent DISPLAYFORM5 by the Chernoff theorem. (See Appendix A for details.) In the rest of this paper, we assume is small. Definition 1. MatrixB is the Divergence Transition Matrix (DTM) of a joint probability DISPLAYFORM0 The singular values ofB satisfy that DISPLAYFORM1 be the left and right singular vectors ofB. Define functions DISPLAYFORM2 for each i = 1,..., K − 1. BID18 further proved that f * i and g * i are solutions to the maximal HGR correlation problem studied by BID12 BID10; BID23, defined as follows: DISPLAYFORM3 The maximal HGR problem finds the K strongest, independent modes in P XY from data. It can be solved efficiently using the Alternating Conditional Expectation (ACE) algorithm with provable error bound (see Appendix B). BID13 further showed that f * and g * are the universal minimum error probability features in the sense that they can achieve the smallest error probability over all possible inference tasks. In this section, we present a performance metric of a given feature representation for a learning task. For a classification task involving input variable X and label Y, most learning algorithms work by finding a k-dimensional functional representation f (x) of X that is most discriminative for the classification. To measure how effective f (x) is in predicting Y, rather than train the model via gradient descent and evaluate its accuracy, we present an analytical approach based on the definition below: Definition 2. Given data matrix X ∈ R m×d and label Y ∈ {1, . . ., |Y|}. Let f (x) be a kdimensional, zero-mean feature function. The H-Score of f (x) with respect to the learning task represented by P Y X is: DISPLAYFORM0 This definition is intuitive from a nearest neighbor search perspective. i.e. a high H-score implies the inter-class variance cov(E P X|Y [f (X)|Y ]) of f is large, while feature redundancy tr(cov(f (X))) is small. Such a feature is discriminative and efficient for learning label Y. More importantly, H(f) has a deeper operational meaning related to the asymptotic error probability for a decision rule based on f in the hypothesis testing context, discussed in the next section. Without loss of generality, we consider the binary classification task as a hypothesis testing problem defined in Section 2.1, with P 1 = P X|Y =0, and P 2 = P X|Y =1. For any k-dimensional feature representation f (x), we can quantify its performance with respect to the learning task using its error DISPLAYFORM0 See Appendix C for the proof. The above theorem shows that H-score H(f) is proportional to the error exponent of the decision region based on f (x) when f (x) is zero-mean with identity covariance. To compute the H-score of arbitrary f, we can center the features f S (x) − E[f S (x)], and incorporate normalization into the computation of the error exponent, which in Definition 2. The details are presented in Appendix D.The proof for Theorem 1 uses the fact that H(f) = B Ξ 2 F, whereB is the DTM matrix, Ξ [ξ 1 · · · ξ k] is the matrix composed of information vectors ξ i and c is a constant. This allows us to infer an upper bound for the H-score of a given learning task: The first inequality is achieved when Ξ is composed of the right singular vectors ofB, i.e. DISPLAYFORM1 DISPLAYFORM2 The corresponding feature functions f opt (X)is in fact the same as the universal minimum error probability features from the maximum HGR correlation problem. The final inequality in Corollary 1 is due to the fact all singular values ofB are less than or equal to 1. Next, we apply H-score to efficiently measure the effectiveness of task transfer learning. We will also discuss how this approach can be used to solve the source task selection problem. A typical way to transfer knowledge from the source task T S to target task T T is to train the target task using source feature representation f S (x). In a neural network setting, this idea can be implemented by copying the parameters from the first N layers in the trained source model to the target model, assuming the model architecture on those layers are the same. The target classifier then can be trained while freezing parameters in the copied layers FIG1 ). Under this model, a natural way to quantify transferability is as follows: Definition 3 (Task transferability). Given source task T S and target task T T, and trained source feature representation f S (x), the transferability from T S to T T is T(S, T) DISPLAYFORM0, where f Topt (x) is the minimum error probability feature of the target task. The statement T(S, T) = r means the error exponent of transfering from T S via feature representation f S is 1 r of the optimal error exponent for predicting the target label Y T. This definition also implies 0 ≤ T(S, T) ≤ 1, which satisfies the data processing inequality if we consider the transferred feature f S (X) as post-processing of input X for solving the target task. And it can not increase information about predicting the target task T. A common technique in task transfer learning is fine-tuning, which adds before the target classifier additional free layers, whose parameters are optimized with respect to the target label. For the operational meaning of transferability to hold exactly, we require the fine tuning layers consist of only linear transformations, such as the model illustrated in Figure 2.a. It can be shown that under the local assumption, H-score is equivalent to the log-loss of the linear transfer model up to a constant offset (Appendix E). Nevertheless, later we will demonstrate empirically that this transferability metric can still be used for comparing the relative task transferability with fine-tuning. With a known f, computing H-score from m sample data only takes O(mk 2) time, where k is the dimension of f (x) for k < m. The majority of the computation time is spent on computing the sample covariance matrix cov(f (X)).The remaining question is how to obtain f Topt efficiently. We use the fact that DISPLAYFORM0, where f and g are the solutions of the HGR-Maximum Correlation problem. This problem can be solved efficiently using the ACE algorithm for discrete variable X. For a continuous random variable X, we can obtain f opt through a different formulation of the HGR maximal correlation problem: DISPLAYFORM1 DISPLAYFORM2 This is also known as the soft HGR problem studied by BID26, who reformulated the original maximal HGR correlation objective to eliminate the whitening constraints while having theoretically equivalent solution. In practice, we can utilize neural network layers to model functions f and g, as shown in Figure 2.b. Given two branches of k output units for both f and g, the loss function can be evaluated in O(mk 2), where m is the batch size. BID18 showed that the sample complexity of ACE is only 1/k of the complexity of estimating P Y,X directly. This also applies to the soft HGR problem due to their theoretical equivalence. Hence transferability can be computed with much less samples than actually training the transfer network. It's also worth noting that, when f is fixed, maximizing the objective in Equation 3 with respect to zero-mean function g in the definition of H-score. In many cases though, the computation of H T (f opt) can even be skipped entirely, such as the problem below: Definition 4 (Source task selection). Given N source tasks T S1,..., T S N with labels Y S1,..., Y S N and a target task T T with label Y T. Let f S1,..., f S N be the minimum error probability feature functions of the source tasks. Find the source task T Si that maximizes the testing accuracy of predicting Y T with feature f Si.We can solve this problem by selecting the source task with the largest transferability to T T. In fact, we only need to compute the numerator in the transferability definition since the denominator is the same for all source tasks, i.e. DISPLAYFORM3 Under the local assumption that P X|Y ∈ N (P X), we can show that mutual information DISPLAYFORM0 (See Appendix F for details.) Hence H-score is related to mutual information by H(f (x)) ≤ 2I(X; Y) for any zero-mean features f (x) satisfying the aforementioned conditions. Figure 3 compares the optimal H-score of a synthesized task when |Y| = 6 with the mutual information between input and output variables, when the feature dimension k changes. The value of H-score increases as k increases, but reaches the upper bound when k ≥ 6, since the rank of the joint probability between X and Y T, as well as the rank of its DTM is 6. As expected, the H-score values are below 2I(X; Y), with a gap due to the constant o. This relationship shows that H-score is consistent with mutual information with a sufficiently large feature dimension k. In practice, H-score is much easier to compute than mutual information when the input variable X (or f S (X)) is continous, as mutual information are either computed based on binning, which has extra bias due to bin size, or more sophisticated methods such as kernel density estimation or neural networks BID8 ). On the other hand, H-score only needs to estimate conditional expectations, which requires less samples. In this section, we validate and analyze our transferability metric through experiments on real image data.1 The tasks considered cover both object classification and non-classification tasks in computer vision, such as depth estimation and 3D (occlusion) edge detection. To demonstrate that our transferability metric is indeed a suitable measurement for task transfer performance, we compare it with the empirical performance of transfering features learned from ImageNet 1000-class classification BID16 ) to Cifar 100-class classification BID15 ), using a network similar to FIG1. Comparing to ImageNet-1000, Cifar-100 has smaller sample size and its images have lower resolution. Therefore it is considered to be a more challenging task than ImageNet, making it a suitable case for transfer learning. In addition, we use a pretrained ResNet-50 as the source model due to its high performance and regular structure. Validation of H-score. The training data for the target task in this experiemnt consists of 20, 000 images randomly sampled from Cifar-100. It is further split 9:1 into a training set and a testing set. The transfer network was trained using stochastic gradient descent with batch size 20, 000 for 100 epochs. FIG3.a compares the H-score of transferring from five different layers (4a-4f) in the source network with the target log-loss and test accuracy of the respective features. As H-score increases, log-loss of the target network decreases almost linearly while the training and testing accuracy increase. Such behavior is consistent with our expectation that H-score reflects the learning performance of the target task. We also demonstrated that target sample size does not affect the relationship between H-score and log-loss FIG3.b).This experiment also showed another potential application of H-score for selecting the most suitable layer for fine-tuning in transfer learning. In the example, transfer performance is better when an upper layer of the source networks is transferred. This could be because the target task and the source task are inherently similar such that the representation learned for one task can still be discriminative for the other. Validation of Transferability. We further tested our transferability metric for selecting the best target task for a given source task. In particular, we constructed 4 target classification tasks with 3, 5, 10, and 20 object categories from the Cifar-100 dataset. We then computed the transferability from ImageNet-1000 (using the feature representation of layer 4f) to the target tasks. The are compared to the empirical transfer performance trained with batch size 64 for 50 epochs in Figure4.c. We observe a similar behavior as the H-score in the case of a single target task in FIG3.a, showing that transferability can directly predict the empirical transfer performance. In this experiment, we solve the source task selection problem for a collection of 3D sceneunderstanding tasks using the Taskonomy dataset from. In the following, we will introduce the experiment setting and explain how we adapt the transferability metric to pixel-to-pixel recognition tasks. Then we compare transferability with task affinity, an empirical transferability metric proposed by.Data and Tasks. The Taskonomy dataset contains 4,000,000 images of indoor scenes of 600 buildings. Every image has annotations for 26 computer vision tasks. We randomly sampled 20, 000 images as training data. Eight tasks were chosen for this experiment, covering both classifications and lower-level pixel-to-pixel tasks. Table 6 summaries the specifications of these tasks and sample outputs are shown in FIG4. For classification tasks, H-score can be easily calculated given the source features. But for pixel-topixel tasks such as Edges and Depth, their ground truths are represented as images, which can not be quantized easily. As a workaround, we cluster the pixel values in the ground truth images into a palette of 16 colors. Then compute the H-score of the source features with respect to each pixel, before aggregating them into a final score by averaging over the whole image. We ran the experiment on a workstation with 3.40 GHz ×8 CPU and 16 GB memory. Each pairwise H-score computation finished in less than one hour including preprocessing. Then we rank the source tasks according to their H-scores of a given target task and compare the ranking with that in. Pairwise Transfer Results. Source task ranking using transferability and affinity are visualized side by side in Figure 7, with columns representing source tasks and rows representing target tasks. For classification tasks (the bottom two rows in the transferability matrix), the top two transferable source tasks are identical for both methods. The best source task is the target task itself, as the encoder is trained on a task-specific network with much larger sample size. Scene Class. and Object Class. are ranked second for each other, as they are semantically related. Similar observations can be found in 2D pixel-to-pixel tasks (top two rows). The on lower rankings are noisier. A slightly larger difference between the two rankings can be found in 3D pixel-to-pixel tasks, especially 3D Occlusion Edges and 3D Keypoints. Though the top four ranked tasks of both methods are exactly the four 3D tasks. It could indicate that these low level vision tasks are closely related to each other so that the transferability among them are inherently ambiguous. We also computed the ranking correlations between transferability and affinity using Spearman's R and Discounted Cumulative Gain (DCG). Both criterion show positive correlations for all target tasks. The correlation is especially strong with DCG as higher ranking entities are given larger weights. The above observations inspire us to define a notion of task relatedness, as some tasks are frequently ranked high for each other. Specifically, we represent each task with a vector consisting of H-scores of all the source tasks, then apply agglomerative clustering over the task vectors. As shown in the dendrogram in Figure 7, 2D tasks and most 3D tasks are grouped into different clusters, but on a higher level, all pixel-to-pixel tasks are considered one category compared to the classifications tasks. Higher Order Transfer. Sometimes we need to combine features from two or more source tasks for better transfer performance. A common way to combine features from multiple models in deep neural networks is feature concatenation. For such problems, our transferability definition can be easily adapted to high order feature transfer, by computing the H-score of the concatenated features. Figure 8 shows the ranking of all combinations of source task pairs for each target task. For all tasks except for 3D Occlusion Edges and Depth, the best seond-order source feature is the combination of the top two tasks of the first-order ranking. We examine the exception in Figure 9, by visualizing the pixel-by-pixel H-scores of first and second order transfers to Depth using a heatmap (lighter color implies a higher H-score). Note that different source tasks can be good at predicting different parts of the image. The top row shows the of combining tasks with two different "transferability patterns" while the bottom row shows those with similar patterns. Combining tasks with different transferability patterns has a more significant improvement to the overall performance of the target task. Transfer learning. Transfer learning can be devided into two categories: domain adaptation, where knowledge transfer is achieved by making representations learned from one input domain work on a different input domain, e.g. adapt models for RGB images to infrared images BID27 ); and task transfer learning, where knowledge is transferred between different tasks on the same input domain BID25 ). Our paper focus on the latter prolem. Empirical studies on transferability. compared the transfer accuracy of features from different layers in a neural network between image classification tasks. A similar study was performed for NLP tasks by BID5. determined the optimal transfer hierarchy over a collection of perceptual indoor scene understanidng tasks, while transferability was measured by a non-parameteric score called "task affinity" derived from neural network transfer losses coupled with an ordinal normalization scheme. Task relatedness. One approach to define task relatedness is based on task generation. Generalization bounds have been derived for multi-task learning BID1 ), learning-to-learn BID19 ) and life-long learning BID21 ). Although these studies show theoretical on transferability, it is hard to infer from data whether the assumptions are satisfied. Another approach is estimating task relatedness from data, either explicitly BID3; Zhang Representation learning and evaluation. Selecting optimal features for a given task is traditionally performed via feature subset selection or feature weight learning. Subset selection chooses features with maximal relevance and minimal redundancy according to information theoretic or statistical criteria BID20; ). The feature weight approach learns the task while regularizing feature weights with sparsity constraints, which is common in multi-task learningLiao & Carin FORMULA9; BID0. In a different perspective, BID13 consider the universal feature selection problem, which finds the most informative features from data when the exact inference problem is unknown. When the target task is given, the universal feature is equivalent to the minimum error probability feature used in this work. In this paper, we presented H-score, an information theoretic approach to estimating the performance of features when transferred across classification tasks. Then we used it to define a notion of task transferability in multi-task transfer learning problems, that is both time and sample complexity efficient. The ing transferability metric also has a strong operational meaning as the ratio between the best achievable error exponent of the transferred representation and the minium error exponent of the target task. Our transferability score successfully predicted the performance for transfering features from ImageNet-1000 classification task to Cifar-100 task. Moreover, we showed how the transferability metric can be applied to a set of diverse computer vision tasks using the Taskonomy dataset. In future works, we plan to extend our theoretical to non-classification tasks, as well as relaxing the local assumptions on the conditional distributions of the tasks. We will also investigate properties of higher order transferability, developing more scalable algorithms that avoid computing the H-score of all task pairs. On the application side, as transferability tells us how different tasks are related, we hope to use this information to design better task hierarchies for transfer learning. DISPLAYFORM0 x m with the following hypotheses: DISPLAYFORM1 Let P x m be the empirical distribution of the samples. The optimal test, i.e., the log likelihood ratio test can be stated in terms of information-theoretic quantities as follows: DISPLAYFORM2 Figure 10: The binary hypothesis testing problem. The blue curves shows the probility density functions for P 1 and P 2. The rejection region and the acceptance region are highlighted in red and blue, respectively. The vertical line indicates the decision threshold. Further, using Sannov's theorem, we have that asymptotically the probability of type I error DISPLAYFORM3 where P * DISPLAYFORM4 m log T } denotes the rejection region. Similarly, for type II error DISPLAYFORM5 where P * 2 = argmin P ∈A D(P ||P 2) and A = {x m : FIG1 The overall probability of error is P (m) e = αP r(H 0) + βP r(H 1) and the best achievable exponent in the Bayesian probability of error (a.k.a. Chernoff exponent) is defined as: DISPLAYFORM6 DISPLAYFORM7 See Cover & BID6 for more information on error exponents and its related theorems. Under review as a conference paper at ICLR 2019 Now consider the same binary hypothesis testing problem, but with the local constraint DISPLAYFORM0 denote the information vectors corresponding to P i for i = 1, 2.Makur et al. FORMULA3 uses local information geometry to connect the error exponent in hypothesis testing to the length of certain information vectors, summarized in the following two lemmas. Lemma 1. Given zero-mean, unit variance feature function f (x): X → R, the optimal error exponent (a.k.a. Chernoff exponent) of this hypothesis testing problem is DISPLAYFORM1 Lemma 2. Given zero-mean, unit variance feature function f (x): X → R, the error exponent of a mismatched decision function of the form l = DISPLAYFORM2 where ξ(x) = P 0 (x)f (x) is the feature vector associated with f (x).As our discussion of transferability mostly concerns with multi-dimensional features, we present the k-dimensional generalization of Lemma 2 below: (Equation 1 in the main paper.) DISPLAYFORM3 for all i, and cov(f (X)) = I, we define a k-d statistics of the form l k = (l 1, ..., l k) where DISPLAYFORM4 be the corresponding feature vectors with DISPLAYFORM5 Proof. According to Cramér's theorem, the error exponent under P i is DISPLAYFORM6. With the techniques developed in local information geometry, the above above problem is equivalent to the following problem: DISPLAYFORM7, it is easy to show that E 1 (λ) = E 1 (λ) when λ = 1 2. Then the overall error probability has the exponent as shown in Equation. Given random variables X and Y, the HGR maximal correlation ρ(X; Y) defined in Equation 2 is a generalization of the Pearson's correlation coefficient to capture non-linear dependence between random variables. According to BID23, it satisfies all seven natural postulates of a suitable dependence measure. Some notable properties are listed below: When the feature dimension is 1, the solution of the maximal HGR correlation is ρ(X; Y) = σ 1, the largest singular value of the DTM matrixB. For k-dimensional features, ρ(X; Y) = k i (σ i). However, computingB requires estimating the joint probability P Y X from data, which is inpractical in real applications. BID4 proposed an efficient algorithm, alternating condition expectation (ACE), inspired by the power method for computing matrix singular values. Require: training samples {(( DISPLAYFORM0 In Algorithm 1, we first initialize g as a random k-dimensional zero-mean function. Then iteratively update f (x) and g(y) for all x ∈ X and y ∈ Y. The conditional distributions on Line 4 and 6 are computed as the empirical average over m samples. The normalization steps on Lines 5 and 7 can also be implemented using the Gram-Schmidt process. Note that the ACE algorithm has several variations in previous works, including a kernel-smoothed version BID4 ) and a parallel version with improved efficiency BID13 ). An alternative formulation that supports continuous X and large feature dimension k has also been proposed recently BID26 ).Next we look at the convergence property of the ACE algorithm. Let f (X), g(Y) be the true maximal correlation functions, and letf (X),g(Y) be estimations computed with Algorithm 1 from m i.i.d. sampled training data. Similarly, denote by DISPLAYFORM1 ] the true and estimated maximal correlations, respectively. Using Sanov's Theorem, we can show that for a small ∆ > 0, the probability that the ratio between the true and estimated maximal correlation is within 1 ± ∆ drops exponentially as the number of samples increases. Hence the ACE algorithm converges in exponential time. The following theorem gives the precise sampling complexity for k = 1. Theorem 2. For any random variables X and Y with joint probability distribution P Y X, if X and Y are not independent, then for any f: X → R and g: DISPLAYFORM2 for any given ∆ > 0. To simplify the proof, we first consider the case when the feature function is 1-dimensional.i.e. f: X → R. We have the following lemma: DISPLAYFORM0 where ξ is the feature vector corresponding to f.Proof. Since ξ(x) = P X (x)f (x), we have DISPLAYFORM1 The last equality uses the assumption that E[f (x)] = 0.Theorem 3 (1D version of Theorem 1). Given P X|Y =0, P X|Y =1 ∈ N X (P 0,X) and features f: DISPLAYFORM2 we first derive the following properties of the conditional expectations of f (x): DISPLAYFORM3 On the R.H.S. of Equation 5, we apply Lemma 4 to write B ξ DISPLAYFORM4 Next consider the L.H.S. of the equation, by Lemma 2, we have DISPLAYFORM5 2 for some constant c 0. DISPLAYFORM6 For k ≥ 2, Lemma 4 can be restated as follows: DISPLAYFORM7 where columns of Ξ are the information vectors corresponding to f 1 (x),..., f k (x).Proof. First, note that DISPLAYFORM8 T where 1 is a column vector with all entries 1 and length |Y|. Since E[f (X)] = 0, we havẽ DISPLAYFORM9 It follows that DISPLAYFORM10 Finally, we derive the multi-dimensional case for Theorem 1.Proof of Theorem 1. Using Lemma 5 and a similar argument as in the simplified proof, the R.H.S of the equation becomes DISPLAYFORM11 By Lemma 3, the L.H.S. of the equation can be written as DISPLAYFORM12 Equation FORMULA51 gives a more understandable expression of the normalization term. We can also writẽ BΞ as follows:B DISPLAYFORM13 T where 1 is a column vector with all entries 1 and length |Y|, we have DISPLAYFORM14 On the other hand, DISPLAYFORM15 The last equality is derived by substituting and into. In softmax regression, given m training examples {( DISPLAYFORM0, the cross-entropy loss of the model is DISPLAYFORM1 Minimizing is equivalent to minimizing D P Y X ||P X Q Y |X where P Y X is the joint empirical distribution of (X, Y).Using information geometry, it can be shown that under a local assumption DISPLAYFORM2 which reveals a close connection between log loss and the modal decomposition ofB. In consequence, it is reasonable to measure the classification performance with B − ΨΦ T 2 F given a pair of (f, θ) associated with (Ψ, Φ).In the context of estimating transferability, we are interested in a one-sided problem, where Φ S is given by the source feature and Ψ becomes the only variable. Training the network is equivalent to finding the optimal weight Ψ * that minimizes the log-loss. By taking the derivative of the Objective function with respect to Ψ, we get DISPLAYFORM3 Substituting FORMULA3 in the Objective of FORMULA3, we can derive the following close-form solution for the log loss. DISPLAYFORM4 The first term in is fixed given T T while the second term has exactly the form of H-score. This implies that log loss is negatively linearly related to H-score. We demonstrates this relationship experimentally, using a collection of synthesized tasks FIG1 ). In particular, the target task is generated based on a random stochastic matrix P Y0|X, and 20 source tasks are generated with the conditional probability matrix P Yi|X = P Y0|X + iλI for some positive constant λ. The universal minimum error probability features for each source task are used as the source features f Si (x), while the respective log-loss are obtained through training a simple neural network in Figure 2 with cross-entropy loss. The relationship is clearly linear with a constant offset. Proposition 1. Under the local assumption that DISPLAYFORM0, whereB of the DTM matrix of X and Y.Proof. First, we define φ DISPLAYFORM1 and let Φ X|Y ∈ R |X |×|Y| denote its matrix version. Then we have DISPLAYFORM2 Next, we express the mutual information in terms of information vector φ X|Y, Here we present some detailed on the comparison between H-score and the affinity score in for pairwise transfer. DISPLAYFORM3 The of the classification tasks are shown in FIG1 and the of Depth is shown in 13.We can see in general, although affinity and transferability have totally different value ranges, they tend to agree on the top few ranked tasks. During the quantization process of the pixel-to-pixel task labels (ground truth images), we are primarily concerned with two factors: computational complexity and information loss. Too much information loss will lead to bad approximation of the original problems. On the other hand, having little information loss requires larger cluster size and computation cost. Figure FORMULA3 shows that even after quantization, much of the information in the images are retained. To test the sensitivity of the cluster size, we used cluster centroids to recover the ground truth image pixel-by-pixel. The 3D occlusion Edge detection on a sample image is shown in FIG1. When the cluster number is set to N = 5 (right), most detected Edges in the ground truth image (left) are lost. We found that N = 16 strikes a good balance between recoverability and computation cost.

represent the short summary for finding the corresponding scientific article (long)

We present a provable and easily-computable evaluation function that estimates the performance of transferred representations from one learning task to another in task transfer learning.

An empirical study of variational inference based on chi-square divergence minimization, showing that minimizing the CUBO is trickier than maximizing the ELBO


Question: Represent the text to find a very short summary (1-2 sentences max)

Question: This paper presents a generic framework to tackle the crucial class mismatch problem in unsupervised domain adaptation (UDA) for multi-class distributions. Previous adversarial learning methods condition domain alignment only on pseudo labels, but noisy and inaccurate pseudo labels may perturb the multi-class distribution embedded in probabilistic predictions, hence bringing insufficient alleviation to the latent mismatch problem. Compared with pseudo labels, class prototypes are more accurate and reliable since they summarize over all the instances and are able to represent the inherent semantic distribution shared across domains. Therefore, we propose a novel Prototype-Assisted Adversarial Learning (PAAL) scheme, which incorporates instance probabilistic predictions and class prototypes together to provide reliable indicators for adversarial domain alignment. With the PAAL scheme, we align both the instance feature representations and class prototype representations to alleviate the mismatch among semantically different classes. Also, we exploit the class prototypes as proxy to minimize the within-class variance in the target domain to mitigate the mismatch among semantically similar classes. With these novelties, we constitute a Prototype-Assisted Conditional Domain Adaptation (PACDA) framework which well tackles the class mismatch problem. We demonstrate the good performance and generalization ability of the PAAL scheme and also PACDA framework on two UDA tasks, i.e., object recognition (Office-Home,ImageCLEF-DA, andOffice) and synthetic-to-real semantic segmentation (GTA5→CityscapesandSynthia→Cityscapes). Unsupervised domain adaptation (UDA) aims to leverage the knowledge of a labeled data set (source domain) to help train a predictive model for a unlabeled data set (target domain). Deep UDA methods bring noticeable performance gain to many tasks (; ; ; ; ; a) by exploiting supervision from heterogeneous sources. Some methods exploit maximum mean discrepancy (MMD) or other distribution statistics like central moments (; ;) for domain adaptation. Recently, generative adversarial learning provides a promising alternative solution to UDA problem. Since the labels of the target instances are not given in UDA, adversarial learning scheme for adaptation suffers from the cross-domain misalignment, where the target instances from a class A are potentially misaligned with source instances from another class B. Inspired by the pseudo-labeling strategy from semi-supervised learning, previous methods either used the pseudo labels in the target domain to perform joint distribution discrepancy minimization (; or developed conditional adversarial learning methods that involve one high-dimensional domain discriminator or multiple discriminators (b;). Though effective, these conditional domain adversarial learning methods align different instances from different domains relying only on their own predictions. Simple probabilistic predictions or pseudo labels may not accurately represent the semantic information of input instances, misleading the alignment. A toy example is given in Fig. 1(a). The pseudo label of the chosen instance x is inclined to be class'square' while the ground truth label is class'circle'. Only guided by the instance prediction, the'circle' class in the target domain and the'square' class in the source domain are easily confused, causing the misalignment in the adversarial domain adaptation. To remedy the misalignment, we propose to exploit the class prototypes for adversarial domain alignment, instead of using only the possibly inaccurate predictions. Prototypes are global feature representations of different classes and are relevant to the inherent semantic structures shared across (a) conditional adversarial learning (b) prototype-assisted adversarial learning Figure 1: Illustration of two adversarial learning schemes. Different from class-agnostic adversarial learning that pursues the marginal distribution alignment but ignores the semantic consistency, (a) conditional adversarial learning relies heavily on the instance-level pseudo labels to perform conditional distribution alignment, while (b) our prototype-assisted adversarial learning integrates the instance-level pseudo labels and global class prototypes to make the conditional indicators more reliable. Class information is denoted in different shapes with source in solid and target in hollow. domains. As shown in Fig. 1(b), class prototypes are expected to remedy the negative effects of inaccurate probabilistic predictions. Motivated by this, we propose a Prototype-Assisted Adversarial Learning (PAAL) scheme which complements instance predictions with class prototypes to obtain more reliable conditional information for guiding the source-target feature representation alignment. Specifically, we summarize the class prototypes from all instances according to their predictions. In this way, on one hand, we lower the dependence of class prototypes on instance predictions which may be inaccurate, and on the other hand, we encourage the instances with greater certainty to contribute more to their corresponding class prototypes. The prototypes are updated dynamically through a moving average strategy to make them more accurate and reliable. Then by broadcasting class prototypes to each instance according to its probability prediction, the inaccurate semantic distribution depicted by instance predictions can be alleviated. Based on reliable prototype-based conditional information, we align both the instance feature representations and the class prototypes through the proposed PAAL scheme to relieve the alignment among semantically dissimilar instances. However, such a conditional domain alignment may promote the confusion among semantically similar instances across domains to some degree. To further alleviate it, we introduce an intra-class objective in the target domain to pursue the class compactness. Built on the proposed PAAL scheme and this intra-class compactness objective, we develop a Prototype-Assisted Conditional Domain Adaptation (PACDA) framework for solving UDA problems. Extensive experimental evaluations on both object recognition and semantic segmentation tasks clearly demonstrate the advantages of our approaches over previous state-of-the-arts; ). The contributions of this work can be summarized into three folds: 1) To the best of our knowledge, we are the first to leverage the class prototypes in conditional adversarial learning to prevent the misalignment in UDA; 2) We propose a simple yet effective domain adversarial learning framework PACDA to remedy the misalignment among semantically similar instances as well as semantically dissimilar instances; 3) The proposed PAAL scheme and PACDA framework are generic, and our framework achieves the state-of-the-art on several unsupervised domain adaptation tasks including object recognition and semantic segmentation. Unsupervised Domain Adaptation. UDA is first modeled as the covariate shift problem where marginal distributions of different domains are different but their conditional distributions are the same. To address it, (Dudík et al., 2006;) exploit a nonparametric instance re-weighting scheme. Another prevailing paradigm (; ;) aims to learn feature transformation with some popular cross-domain metrics, e.g., the empirical maximum mean discrepancy (MMD) statistics. Recently, a large number of deep UDA works (; ; ;) have been developed and boosted the performance of various vision tasks. Generally, they can be divided into discrepancy-based and adversarial-based methods. Discrepancy-based methods address the dataset shift by mitigating specific discrepancies defined on different layers of a shared model between domains, e.g. resembling shallow feature transforma-tion by matching higher moment statistics of features from different domains . Recently, adversarial learning has become a dominantly popular solution to domain adaptation problems. It leverages an extra domain discriminator to promote domain confusion. designs a gradient reversal layer inside the classification network and utilizes an inverted label GAN loss to fool the discriminator. Pseudo-labeling. UDA can be regarded as a semi-supervised learning (SSL) task where unlabeled data are replaced by the target instances. Therefore, some popular SSL strategies, e.g., entropy minimization (; b), mean-teacher , and virtual adversarial training , have been successfully applied to UDA. Pseudo-labeling is favored by most UDA methods due to its convenience. For example, (; exploit the intermediate pseudo-labels with tri-training and self-training, respectively. obtains target-specific prototypes with the help of pseudo labels and aligns prototypes across domains at different levels. Recently, curriculum learning , self-paced learning and re-weighting schemes ) are further leveraged to tackle possible false pseudo-labels. Conditional Domain Adaptation. Apart from the explicit integration with the last classifier layer, pseudo-labels can also be incorporated into adversarial learning to enhance the feature-level domain alignment. Concerning shallow methods , pseudo-labels can help mitigate the joint distribution discrepancy via minimizing multiple class-wise MMD measures. proposes to align the joint distributions of multiple domain-specific layers across domains based on a joint maximum mean discrepancy criterion. Recently, (b;) leverages the probabilities with multiple domain discriminators to enable fine-grained alignment of different data distributions in an end-to-end manner. In contrast, conditions the adversarial domain adaptation on discriminative information via the outer product of feature representation and classifier prediction. Motivated by the semantically-consistent GAN, imposes a multi-way adversarial loss instead of a binary one on the domain alignment. However, these methods all highly rely on the localized pseudo-labels to align labelconditional feature distributions and ignore the global class-level semantics. As far as we know, we are the first to exploit class prototypes to guide the domain adversarial learning. Compared with (;, our PACDA framework complements the original feature representations with reliable semantic features and merely involves two low-dimensional domain discriminators, making the domain alignment process simple, conditional, and reliable. In this section, we first begin with the basic settings of UDA and then give detailed descriptions on the proposed PAAL scheme and the PACDA framework. Though proposed for image classification, they can also be easily applied to semantic segmentation. In a vanilla UDA task, we are given label-rich source domain data {( sampled from the joint distribution P s (x s, y s) and unlabeled target domain data {x, where x i s ∈ X S and y i s ∈ Y S denote an image and its corresponding label from the source domain dataset, x i t ∈ X T denotes an image from the target domain dataset and P s = Q t . The goal of UDA is to learn a discriminative model from X S, Y S, and X T to predict labels for unlabeled target samples X T . As described in , a vanilla domain adversarial learning framework consists of a feature extractor network G, a classifier network F, and a discriminator network D. Given an image x, we denote the feature representation vector extracted by G as f = G(x) ∈ R d and the probability prediction obtained by F as p = F (f) ∈ R c where d means the feature dimension and c means the number of classes. The vanilla domain adversarial learning method in can be formulated as optimizing the following minimax optimization problem: min. M ema represents the global class prototype matrix while M s,t is computed by source or target instances within current batch. where the binary domain classifier D: predicts the domain assignment probability over the input features, L y (G, F) is the cross-entropy loss of source domain data as for the classification task, and λ adv is the trade-off parameter. The misalignment in UDA of multi-class distributions challenges the popular vanilla adversarial learning. In previous works (; ;, target domain data are conditioned only on corresponding pseudo labels predicted by the model for adversarial domain alignment. The general optimization process of these methods is the same as aforementioned vanilla domain adversarial learning, except that feature representations jointly with predictions are considered by the discriminator D: is the conditional adversarial loss that leverages the classification predictions p s and p t . A classic previous work implicitly conditions the feature representation on the prediction through the outer product f ⊗ p, and uses one shared discriminator to align the conditioned feature representations. further proves that using the outer product can perform much better than simple concatenation f ⊕ p. Different from, (b;) explicitly utilize multiple class-wise domain discriminators to align the feature representations relying on the corresponding predictions. However, the pseudo labels may be inaccurate due to the domain shift. Therefore, only conditioning the alignment on pseudo labels can not safely remedy the misalignment. Compared with the pseudo labels, the class prototypes are more robust and reliable in terms of representing the shared semantic structures. To acquire more reliable and accurate conditional information for domain adversarial learning, we propose to complement instance predictions with class prototypes and reformulate the adversarial loss to: c×d denotes the global class prototype matrix in our prototype-assisted adversarial learning loss L paal adv (G, D). In reality, the reliable conditional information is obtained through broadcasting the global class prototypes to each independent instance according to its prediction p. We propose to summarize feature representations of the instances within the same class as the corresponding prototype. Then the probability prediction is leveraged to obtain accurate class prototypes. Using predictions as weights can adaptively control the contributions of typical and non-typical instances to the class prototype, making class prototypes more reliable. Specifically, we first gather the feature representation of each instance relying on its prediction to generate the batch-level class prototypes. Then the global class prototypes can be obtained by virtue of an averaging strategy such as exponential moving average (ema) on the batch ones. This process can be formulated as Here n means the batch size, p k,i represents the probability of the i-th instance belonging to the k-th semantic class, λ ema is an empirical weight, M ∈ R c×d is the batch-level class prototype matrix and M ema is the global one computed by certain source domain data and contributes to more reliable conditional information exploited by discriminators. Similarly, batch-level class prototypes are broadcast to each instance in this batch through M T a p a which can be denoted as f a, a ∈ {s, t}. 3.3 PROTOTYPE-ASSISTED CONDITIONAL DOMAIN ADAPTATION (PACDA) FRAMEWORK With our prototype-based conditional information, we further propose a Prototype-Assisted Conditional Domain Adaptation (PACDA) framework. This framework aligns both instance-level and prototype-level feature representations through PAAL and promotes the intra-class compactness in target domain such that the misalignment can be substantially alleviated even though no supervision is available in the target domain. Its overall architecture is shown in Fig. 2. Besides the backbone feature extractor G and the task classifier F, there are two discriminators in our framework PACDA, i.e., the instance-level feature discriminator D f and the prototype-level feature discriminator D p. We can formulate our general objective function as (where λ denotes balance factors among different loss functions, L y is the supervised classification loss on source domain data described by Eq., L f adv is the adversarial loss to align instance feature representations across domains, L p adv is the adversarial loss to align class prototype representations across domains, and L t is the loss to promote the intra-class compactness in target domain. Instance-Level Alignment Conditioning the instance feature representation on our prototype-based conditional information, we seek to align feature representations across domains at the instance-level through discriminator D f. With the assistance of the accurate semantic structures embedded in class prototypes, misalignment among semantically dissimilar instances can be effectively alleviated. We can define the instance-level adversarial loss L adv f as Prototype-Level Alignment Instance-level alignment only implicitly aligns the multi-class distribution across domains, which may not ensure the semantic consistency between two domains. Besides, since in practice global class prototypes are collected from only source domain data, which possibly cannot accurately represent inherent semantic structures in the target domain due to the domain shift. Taking into account these two causes, we perform the prototype-level alignment with discriminator D p to explicitly align the class prototype representations across domains. The specific loss function Intra-Class Compactness Although adversarial alignment based on PAAL can relieve the misalignment among obviously semantically different instances, it cannot well handle the misalignment among semantically similar instances. Specifically, incorporating class prototypes into instance predictions would confuse semantically similar instances during domain alignment and in the misalignment among them. To solve this problem, our framework further promotes the intra-class compactness in the target domain to enlarge the margin between instances of semantically similar classes. Taking the prototypes as proxy, we minimize the following loss for target domain samples to encourage the intra-class compactness: Thus, the complete minimax optimization problem of our PACDA framework can be formulated as With only two low-dimensional (2 × d) discriminators added, we effectively remedy the misalignment in domain adversarial learning. Some theoretical insights with the help of domain adaptation theory is discussed in the Appendix. We conduct experiments to verify the effectiveness and generalization ability of our methods, i.e., PACDA (full) in Eq. and PAAL (λ p adv = λ t = 0) on two different UDA tasks, including cross-domain object recognition on ImageCLEF-DA 1, Office31 and OfficeHome , and synthetic-to-real semantic segmentation for GTA5 →Cityscapes and Synthia → Cityscapes. Datasets. Office-Home is a new challenging dataset that consists of 65 different object categories found typically in 4 different Office and Home settings, i.e., Artistic (Ar) images, Clip Art (Ca), Product images (Pr), and Real-World (Re) images. ImageCLEF-DA is a standard dataset built for the'ImageCLEF2014:domain-adaptation' competition. We follow to select 3 subsets, i.e., C, I, and P, which share 12 common classes. Office31 is a popular dataset that includes 31 object categories taken from 3 domains, i.e., Amazon (A), DSLR (D), and Webcam (W). Cityscapes is a realistic dataset of pixel-level annotated urban street scenes. We use its original training split and validation split as the training target data and testing target data respectively. GTA5 consists of 24,966 densely labeled synthetic road scenes annotated with the same 19 classes as Cityscapes. For Synthia, we take the SYNTHIA-RAND-CITYSCAPES set as the source domain, which is composed of 9,400 synthetic images compatible with annotated classes of Cityscapes. Implementation Details. For object recognition, we follow the standard protocol , i.e. using all the labeled source instances and all the unlabeled target instances for UDA, and report the average accuracy based on three random trials for fair comparisons. ), we experiment with ResNet-50 model pretrained on ImageNet. Specifically, we follow to choose the network parameters, and all convolutional layers and the classifier layer are trained through backpropagation, where λ t =5e-3, λ ema =5e-1, λ f adv and λ p adv increase from 0 to 1 with the same strategy as . Regarding the domain discriminator, we design a simple two-layer classifier (256→1024→1) for both D f and D p. Empirically, we fix the batch size to 36 with the initial learning rate being 1e-4. For semantic segmentation, we adopt DeepLab-V2 (a) based on ResNet-101 as done in (; b;). Following DCGAN , the discriminator network consists of three 4 × 4 convolutional layers with stride 2 and channel numbers {256, 512, 1}. In training, we use SGD to optimize the network with momentum (0.9), weight decay (5e-4), and initial learning rate (2.5e-4). We use the same learning rate policy as in (a). Discriminators are optimized by Adam with momentum (β 1 = 0.9, β 2 = 0.99), initial learning rate (1e-4) along with the same decreasing strategy as above. For both tasks, λ f adv is set to 1e-3 following and λ ema is set to 0.7. For GTA5→Cityscapes, λ p adv =1e-3 and λ t =1e-5. For Synthia→Cityscapes, λ p adv =1e-4 and λ t =1e-4. All experiments are implemented via PyTorch on a single Titan X GPU. The total iteration number is set as 10k for object recognition and 100k for semantic segmentation. For objection recognition tasks, we choose the hyper-parameters which have the minimal mean entropy of target data on Ar→Cl for convenience. For semantic segmentation tasks, training split of Cityscapes is used for the hyper-parameters selection. Data augmentation skills like random scale or random flip and ten-crop ensemble evaluation are not adopted. Cross-Domain Object Recognition. The comparison between our methods (i.e., PAAL and PACDA) and state-of-the-art (SOTA) approaches (; on Office-Home, Office31, and ImageCLEF-DA are shown in Tables 1 and 2, respectively. As indicated in these tables, PACDA improves previous approaches in the average accuracy for all three benchmarks (e.g., 67.3%→68.7% for Office-Home, 88.1%→88.8% for ImageCLEF-DA, and 87.7%→89.3% for Office31). Generally, PACDA performs the best for most transfer tasks. Taking 89.9 78.5 94.7 79.5 92.0 89.7 87.4 90.1 92.5 72.1 98.8 69.9 100. 87.2 CAT 91 Table 3: Comparison of synthetic-to-real semantic segmentation using the same architecture with NonAdapt and AdaptSeg , AdvEnt (b), CLAN and AdaptPatch . Top: GTA5 → Cityscapes. Bottom: Synthia → Cityscapes. a careful look at PAAL, we find that it always beats CDAN and achieves competitive performance with SOTA methods like CAT . Synthetic-to-real Semantic Segmentation. We compare PAAL and PACDA with SOTA methods (; b;) on synthetic-to-real semantic segmentation. Following (b), we evaluate models on all 19 classes for GTA5→Cityscapes while on only 13 classes for Synthia→Cityscapes. As shown in Table 3, without bells and whistles, our PAAL method outperforms all of those methods and our PACDA framework further achieves new SOTA on both tasks, i.e., 43.8%→46.6% for GTA5→Cityscapes and 47.8%→49.2% for Synthia→Cityscapes in terms of the mean IoU (mIoU) value. Quantitative Analysis. To verify the effectiveness of each component in Eq., we introduce a variant named PAAL f,p that merely ignores the intra-class objective (λ t = 0). The empirical convergence curves about Ar→Cl in Fig.(a) imply that all of our variants tend to converge after 10k iterations, and the second term can help accelerate the convergence. Fig. 3(b) shows that all terms in the PACDA framework, i.e., PAAL alignment at different levels and the intra-class objective, can bring evident improvement on both tasks. As shown in Fig.(c), we provide the proxy A-distances of different methods for Ar→Cl and C→I. The A-distance Dist A =2(1 − 2) is a popular measure for domain discrepancy, where is the test error of a binary classifier trained on the learned features. All the UDA methods have smaller distances than'source only' by aligning different domains. Besides, our PACDA has the minimum distance for both tasks, implying that it can learn better features to bridge the domain gap between domains. To testify the sensitivity of our PACDA, in Fig. Qualitative Analysis. For object recognition, we study the t-SNE visualizations of aligned features generated by different UDA methods in Fig. 4. As expected, conditional methods including CDAN and PAAL can semantically align multi-class distributions much better than DANN. Besides, PAAL learns slightly better features than CDAN due to less misalignment. Once considering the intra-class objective, PACDA further enhances PAAL by pushing away semantically confusing classes, which achieves the best adaptation performance. For semantic segmentation, we present some qualitative in Fig. 5. Similarly, PAAL effectively improves the adaptation performance and PAAL f,p as well as PACDA can further improve the segmentation . In this work, we developed the prototype-assisted adversarial learning scheme to remedy the misalignment for UDA tasks. Unlike previous conditional ones whose performance is vulnerable to inaccurate instance predictions, our proposed scheme leverages the reliable and accurate class prototypes for aligning multi-class distributions across domains and is demonstrated to be more effective to prevent the misalignment. Then we further augment this scheme by imposing the intra-class compactness with the prototypes as proxy. Extensive evaluations on both object recognition and semantic segmentation tasks clearly justify the effectiveness and superiority of our UDA methods over well-established baselines. We try to explain why our PAAL works well for UDA according to the domain adaptation theory proposed in . Denote by P (F) = E (f,y)∈P [F (f) = y] the risk of a classifier model F ∈ H w.r.t. the distribution P, and by P (F 1, F 2) = E (f,y)∈P [F 1 (f) = F 2 (f)] the disagreement between hypotheses F 1, F 2 ∈ H. Particularly, gives a well-known upper bound on the target risk Q (F) of classifier F in the following, where F * is the ideal classifier induced from F * = arg min F ∈H [P (F) + Q (F)], and the last term is related to the classical H-divergence d H∆H (P, Q) = 2 sup F,F * ∈H | P (F, F *) − Q (F, F *)|. Besides, according to , the empirical H-divergence calculated by m respective samples from distributions P and Q converges uniformly to the true H-divergence for classifier classes H of finite VC dimension d, which is expressed as d H∆H (P, Q) ≤d H∆H (P, Q) + 4 d log(2m) + log(2/δ) m. The work introduces a binary domain discriminator to minimize the empirical H-divergenced H∆H (P, Q), which aligns the marginal distributions well. However, if two multi-class distributions P and Q are not semantically aligned, there may not be any classifier with low risk in both domains, which means the second term of the upper bound in Eq. is very large. The proposed PAAL scheme leverages reliable conditional information in the adversarial learning module so that semantically similar samples from different domains are implicitly aligned, thus it has a high possibility of decreasing the second term. Compared with, the input to domain the adversarial learning module is much more compact (2 × d c × d), which helps decrease the second term in Eq..

Represent the short summary for finding the corresponding scientific article (long)

We propose a reliable conditional adversarial learning scheme along with a simple, generic yet effective framework for UDA tasks.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

We propose a framework to learn confidence-calibrated networks by designing a novel loss function that incorporates predictive uncertainty estimated through stochastic inferences.