title: Adversarial Dual Distinct Classifiers for Unsupervised Domain Adaptation
authors: Taotao Jing Zhengming Ding
text: 
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  Adversarial Dual Distinct Classifiers for Unsupervised Domain Adaptation
      Taotao Jing^1,  Zhengming Ding^2
  
  ^1Department of ECE, Indiana University-Purdue University Indianapolis, USA 
  ^2Department of CIT, Indiana University-Purdue University Indianapolis, USA
  
  jingt, zd2@iu.edu
        ;  
  =================================================================================================================================================================================================================
  
  
  
  
  
     Unsupervised Domain adaptation (UDA) attempts to recognize the unlabeled target samples by building a learning model from a differently-distributed labeled source domain. Conventional UDA concentrates on extracting domain-invariant features through deep adversarial networks. However, most of them seek to match the different domain feature distributions, without considering the task-specific decision boundaries across various classes. In this paper, we propose a novel Adversarial Dual Distinct Classifiers Network (AD^2CN) to align the source and target domain data distribution simultaneously with matching task-specific category boundaries. To be specific, a domain-invariant feature generator is exploited to embed the source and target data into a latent common space with the guidance of discriminative cross-domain alignment. Moreover, we naturally design two different structure classifiers to identify the unlabeled target samples over the supervision of the labeled source domain data. Such dual distinct classifiers with various architectures can capture diverse knowledge of the target data structure from different perspectives. Extensive experimental results on several cross-domain visual benchmarks prove the model's effectiveness by comparing it with other state-of-the-art UDA.
  
  
  
  
  
  § INTRODUCTION
  
  
  Deep neural networks (DNNs) have made significant progress with the help of numerous well-labeled training data and achieved remarkable performance improvement on various tasks <cit.>. However, massive amounts of annotated training data are not always available due to the dramatically expensive data collecting and annotating costs. Domain adaptation (DA) has attracted extremely increasing attention because it focuses on a frequent and real-world issue when we have no access to massive labeled target domain training data <cit.>. The mechanism of domain adaptation is to uncover the common latent factors across the source and target domains and reduce both the marginal and conditional mismatch in terms of the feature space between domains. Following this, different domain adaptation techniques have been developed, including feature alignment and classifier adaptation <cit.>.
  
  Recent research efforts on domain adaptation have already shown promising performance via seeking an effective domain-invariant feature extractor across two domains so that the source knowledge could be adapted to facilitate the recognition task in the target domain <cit.>. The idea is to deploy cross-domain matching losses to guide the domain-invariant feature learning. First of all, the discrepancy loss (e.g., maximum mean discrepancy (MMD)) is one of the most widely-used strategies to measure the distribution difference across the source and target domains <cit.>. Along this, many DA approaches explore to design a class-wise MMD by incorporating the pseudo labels of target data <cit.>. Secondly, the adversarial loss has been successfully applied to eliminate the domain shifts on the feature or pixel level <cit.>, where one domain discriminator or more are trained with a feature generator in an adversarial manner. Moreover, various reconstruction penalties are proposed on target samples to obtain the target specific structures, e.g., iCAN <cit.>. However, most existing domain adaptation methods suffer from explicitly matching source and target domains distribution by only considering the domain-wise adaptation while ignoring the alignment of task-specific category boundaries. 
  
  
  
  
  
  
  To address this issue, some recent DA works aim to consider the task-specific category-level alignment jointly <cit.>. Along this line, Saito present Maximum Classifier Discrepancy (MCD) with two task-specific classifiers to detect category boundaries and jointly align features distribution and category boundaries across domains <cit.>. Following this, Lee propose Sliced Wasserstein Discrepancy (SWD) as a new probability distribution discrepancy measurement to capture the natural notion of dissimilarity between the outputs of task-specific classifiers <cit.>. Later on, <cit.> promotes Domain-Symmetric Networks (SymNets) as well as a two-level (feature-level and category-level) domain confusion scheme to drive the learning of intermediate features to be invariant at the corresponding categories of the two domains. These method benefit from various strategies to maximize the disparity of the dual classifiers prediction results, however, considering the utterly same architecture classifiers not only limits the features distribution knowledge obtained from different perspectives, but also suffers from the risk that the two task-specific classifiers may result in the similar class-wise boundaries, especially when the imbalanced data distribution across various categories.  
  
  
  In this paper, we propose a novel Adversarial Dual Distinct Classifiers Network (AD^2CN) with two different-architecture classifiers, e.g., Neural Networks Classifier and Prototypical Classifier, to facilitate the alignment of both domain distributions and category decision boundaries (Fig. <ref>). To our best knowledge, it is a pioneering work to explore dual different structure classifiers in domain adaptation. The general idea is to explore adversarial training over two different architecture classifiers on the output of one domain-invariant feature generator. To sum up, we highlight the three-fold contributions of this paper as follows:
  
  	
    * We exploit dual different architecture task-specific classifiers over source supervision to exploit the task-specific decision boundaries on the target domain. With different properties of dual classifiers in prediction, we have a better chance of capturing ground-truth classifier decision boundaries for the target domain.
      
    * We propose a novel discriminative cross-domain alignment loss and Importance Guided Optimization strategy to mitigate the cross-domain mismatching. This will facilitate the process of aligning the domain-invariant embedding features distribution across domains, and eliminate the distraction of misestimated target samples at the beginning of optimizing.
      
    * We adopt a discrepancy loss to maximally improve the prediction performance of dual classifiers in coupling the cross-domain label distributions, which is trained in an adversarial way with domain-invariant feature generator and dual classifiers. Thus, they can benefit from each other to boost the target learning task.
  
  
  
  
  § RELATED WORK
  
  
  Domain adaptation (DA) has been extensively studied recently, which casts a light when there are no or limited labels in target domain and shows very promising performance in different vision applications <cit.>.
  
  
  With the renaissance of deep neural networks, deep DA methods successfully embed DA into deep learning pipelines by either minimizing an appropriate distribution distance metric <cit.> or leveraging adversarial technologies to generate domain-invariant representations <cit.>. The idea behind this is to incorporate domain alignment strategies at the top layers to explicitly solve the enlarged domain discrepancy resulted from traditional deep learning models. To name a few, Long proposed Domain Adaptation Network (DAN) to incorporate multiple kernel MMD distances across domains among the last three task-specific layers <cit.>. Long presented a joint adaptation network (JAN) as well as a joint MMD criterion <cit.>. Another strategy is to leverage generative adversarial networks (GAN) <cit.> to couple the cross-domain discrepancy in an adversarial manner <cit.>. Such techniques aim to train a domain discriminator to 
  differentiate source and target samples, while the feature generator will deceive the domain discriminator, such that the domain-invariant features will be produced. Ganin proposed DANN to generate task-specific discriminative while domain-wise indiscriminative features <cit.>. Tzeng presented ADDA for adversarial adaptation <cit.>.
  
  
  
  
  
  
  Both discrepancy and adversarial loss based methods attempt to match the whole source and target domain distribution completely, neither of them considers the target domain data structure and task-specific decision boundaries. To address this, Saito adopted the task-specific category decision boundaries and proposes a model with two classifiers as a discriminator to detect the relationship between the source and target domain data (MCD) <cit.>. By maximizing the prediction results of the two classifiers, the framework is able to screen out target samples that are near the category decision boundaries and far from the source domain support. Following this, 
  
  Lee extended MCD and proposed a novel Wasserstein metric to capture the natural notion of dissimilarity between the outputs of two task-specific classifiers <cit.>. Most recently, Li claimed that label distribution alignment is still not enough and present Joint Adversarial Domain Adaptation (JADA) to simultaneously align domain-wise and class-wise distributions across source and target in a unified adversarial learning process <cit.>. Unfortunately, existing works seek to maximize the prediction difference between two same architecture classifiers to explore different task-specific knowledge, limiting the divergence of category decision boundaries captured across domains.
  
  Differently, we propose a novel framework with two different structure classifiers, which assist the model to learn more diverse data distribution patterns and less similar category decision boundaries from different perspectives. Integrating task-specific category boundaries and feature-level cross-domain adaptation, our proposed model is able to narrow the data mismatch of source and target domain in the shared domain invariant embedding space.  Moreover, we explore a cross-domain discriminative distribution alignment under the sample Importance Guided Optimization strategy, which has been experimentally proven to eliminate the source and target domain shift.
  
  
  
  
  
  
  
  § THE PROPOSED METHOD
  
  
  
  
  
  
   §.§ Preliminaries and Motivation
  
  
  Given a labeled source domain 𝒟_s = {𝐗_s, 𝐘_s } = { (𝐱_s^i, 𝐲_s^i)}_i=1^n_s which contains n_s labeled samples, as well as an unlabeled target domain 𝒟_t = {𝐗_t} = {𝐱_t^j }_j=1^n_t of n_t unlabeled samples. 
  
  𝐏_s(𝐱_s) and 𝐏_t(𝐱_t) denote the source and target domain different data distributions respectively (𝐏_s(𝐱_s) ≠𝐏_t(𝐱_t)). 𝒞_s and 𝒞_t mean source and target domain identical label spaces. 𝐘_s ∈ℝ^n_s × C is the source domain ground truth label set which is accessible for training, where C = C_s/t = |𝒞_s/t| is the number of total categories. The goal of domain adaption is to seek a model to predict the unlabeled target data over the supervision from the source domain.
  
  
  
  Recent domain adaptation works apply adversarial networks to generate domain invariant features of the source and target domain samples, which will make the classifiers trained only on the source domain data available on the target domain<cit.>. Most of them aim to match the distribution of source and target domain completely, without considering the task-specific decision boundaries between different categories. Most recently, the idea of dual adversarial classifiers <cit.> has been explored to replace the original adversarial domain adaptation with a binary domain discriminator. However, they obtain two same-type classifiers from scratch over labeled source data. This would limit the discriminative ability in target prediction since the same-type classifiers would tend to have similar properties. Traditional neural networks classifier aims to fit the training data by achieving optimal objective value, thus the learned classifier boundaries would capture the global structure of the data to maximally separate different classes. Such a decision boundary over source supervision cannot be well adapted to target samples in different distribution. Therefore, two same-architecture neural network classifiers over source supervision are challenging to diversify the decision boundaries.
  
  This motivates us to explore two different architecture classifiers, and thus we propose a novel adversarial dual classifiers network with two different structure classifiers, Neural Networks Classifier C_N(·) and Prototypical Classifier C_P(·) <cit.>, which can capture various data distribution pattern and more diverse task-specific category boundaries from different perspectives, and also promote the out of source support target samples detection process.
  Interestingly, the prototypical classifier explores the local structure of the data since prototypes are used to assign labels based on the similarity between samples and each prototype. The competition between two different structure classifiers is more likely to diversify the decision boundaries to benefit from adversarial training with domain-invariant generator.  
  
  
  
  
  
   §.§ Adversarial Dual Distinct Classifiers Network
  
  
  
  
  
  We first present the overall framework of our proposed adversarial dual classifier network in Fig. <ref>. Given the labeled source and unlabeled target domain data, the domain invariant embedding features are generated and aligned by the discriminative cross-domain alignment, then the dual classifiers, which consist of two classifiers with different architectures, will promote the task-specific decision boundaries further. G(·) is the generator used to map source and target domain data to a shared embedding feature space, in which the target samples are close to the support of the source domain data. The following two different structure classifiers, fully-connected neural network classifier C_N(·) and prototypical classifier C_P(·), will capture diverse and various task-specific categories knowledge on target domain from different perspectives.
  
  
  
  
  
  
    §.§.§ Dual Classifiers Over Source Supervision
  
  
  
  Since 𝐗_s and 𝐗_t have different distributions, a domain-invariant feature generator G(·) is deployed to capture more enriched information across source and target through hierarchical structures, followed by our dual classifiers, C_N(·) (fully-connected neural network classifier) and C_P(·) (prototypical classifier). With the extracted feature 𝐳_s/t^i = G(𝐱_s/t^i) from G(·) as input, we can calculate the corresponding probability prediction with two classifiers C_N(·) and C_P(·) as ŷ_N/P,s/t^i=C_N/P(𝐳_s/t^i).
  
  
  
  Specifically, C_N(·) is the traditional multi-layer non-linear classifier, while C_P(·) is defined as the similarity between target sample feature 𝐳_t^i to each category prototype μ_c (i.e., class center), that is, ŷ_P,t^i(c)=Φ(𝐳_t^i,μ_c).
  
  For each class, the prototype μ_c = 1/n_t^c∑_i=1^n_t^c𝐳_t^i(c), where n_t^c and 𝐳_t^i(c) denote the number of target samples and extracted domain invariant feature belonging to class c. We apply the C_P(·) prediction ŷ_P,t^i as the predicted pseudo label to target sample 𝐱_t^i to get the category prototypes μ_c.
  
  
  
  
  In order to obtain task-specific discriminative features from generator G(·), while keeping classification performance on source domain, we add the supervision from source to learn the parameters of C_N(·) and G(·). Since C_P(·) does not contain any trainable parameters, the supervision over C_P(·) prediction on the source domain tends to optimize the generator G(·) only. To this end, we aim to minimize the cross-entropy loss over 𝐘_s and predicted labels from C_N(·) and C_P(·), defined as follows:
  
      ℒ_s = 1/n_s∑_i=1^n_sℒ(ŷ_N,s^i, 𝐲_s^i)+1/n_s∑_i=1^n_sℒ(ŷ_P,s^i, 𝐲_s^i),
  
  where ℒ is the cross-entropy loss. ŷ_N,s^i and ŷ_P,s^i are the probability outputs of classifier C_N(·) and C_P(·), while 𝐲_s^i is the ground-truth label of source sample 𝐱_s^i, respectively.
  
  
  
    §.§.§ Adversarial Dual Classifiers
  
  
  
  
  The dual classifiers are capable of recognizing target domain samples close to the support of the source domain. For those target domain samples which are far from the source domain support, the two classifiers would tend to obtain different probability outputs. To detect target samples outside of the support from source supervision, we propose to measure the disagreement of the classifiers prediction results with distribution discrepancy measurement <cit.>. 
  
  
  
  Existing works exploit varying the dual classifiers by maximizing the divergence between the predictions. However, the same classifier structure with slightly different random initializations <cit.> will weaken the ability to capture diverse task-specific knowledge and decision boundaries from different perspectives.
  
  In our model, we build two different architecture classifiers, which are more likely to capture the inconsistent information from various perspective. Thus, adversarial training would further enhance the target prediction performance, and the classifier discrepancy is defined as:
  
  
      ℒ_dis = ℱ(ŷ^i_N,t,ŷ^i_P,t),
  
  
  where 
  
  ŷ^i_N/P,t represent the probability prediction obtained from the two classifiers for the sample 𝐱^i_t respectively. ℱ(·,·) denotes the discrepancy measurement function, 
  
  which is able to capture distribution geometric information to calculate the discrepancy between the probability prediction distributions, and solve gradient vanishing problems occurred in adversarial learning methods.
  
  
  
    §.§.§ Discriminative Cross-Domain Alignment
  
  
  
  
  So far, our model only aligns cross-domain distributions in terms of label space, we further exploit feature distribution alignment to boost the domain-invariant feature learning. 
  
  Empirical Maximum Mean Discrepancy (MMD) has been verified as a promising technique to minimize the domain-wise mean of two domains or class-wise mean with the pseudo labels of the target <cit.>. The domain-wise MMD to measure marginal distribution across the source and target domains is defined as ℋ(𝔼_𝐱_s^i  ∼𝒟_s[𝐳_s^i]-𝔼_𝐱_t^j  ∼𝒟_t[𝐳_t^j]) <cit.>, where ℋ(·) is the function used to evaluate the distribution difference. Furthermore, existing works <cit.> also seek to explore the class-wise MMD to align conditional distribution disparity across domain:
  
      ℒ_c = 1/C∑_c=1^Cℋ(𝔼_𝐱_s^i  ∼𝒟_s^c[𝐳_s^i]-𝔼_𝐱_t^j  ∼𝒟_t^c[𝐳_t^j]),
  
  where C denotes the total number of categories, 𝐳_s/t^i/j denote the generated embedding representations of source sample 𝐱_s^i and target sample 𝐱_t^j belonging to class c. 
  
  However, conventional DA algorithms only seek to minimize the distribution difference between source and target domains when samples are from the same class. We further propose to explicitly take the information of different categories into account and measure the diff-class divergence across domains defined as:
  
      ℒ_d =  1C1C-1∑_c=1^C∑_c'=1, 
      c' ≠ c^Cℋ(𝔼_𝐱_s^i  ∼𝒟_s^c[𝐳_s^i]-𝔼_𝐱_t^j∼𝒟_t^c'[𝐳_t^j]),
  
  where the diff-class divergence ℒ_d calculates the average distances of all different class center pairs across domains. To sum up, our discriminative cross-domain alignment is defined as ℒ_m = ℒ_c - ℒ_d. 
  
  Due to the lack of target domain labels, we explicitly assign ŷ_P,t^i, the prediction of C_P(·), as pseudo labels to the target samples 𝐱_t^i. To exploit more effective knowledge transfer iteratively, we propose an Importance Guided Optimization strategy to only consider those target samples with high prediction confidences during the cross-domain alignment since lower-confident samples would mislead the optimization. That is, only samples with {(𝐱_t^i, ŷ_P,t^i(c)) |ŷ_P,t^i(c) > σ_1, 𝐱_t^i ∈𝒟_t } are accepted to construct the cross-domain alignment ℒ_m, where ŷ_P,t^i(c) is the C_P(·) probability prediction of 𝐱_t^i belonging to class c, and σ∈ [0,1] is a constant threshold. It is noteworthy that we do not impose always covering the whole label space during training, since only considering those classes with high-confident samples is prone to result in effective cross-domain alignment by avoiding too many mis-classified target samples, especially in the early training stage.
  
  
  
  
  
  
  
  
  
  
  
  
  
  
   §.§ Overall Objective and Optimization
  
  
  
  To eliminate the side effect of uncertainty on unlabeled target prediction, we also explore the entropy minimization regularization <cit.>:
  
      ℒ_em = -1/n_t∑_i=1^n_t∑_c=1^C (ŷ_N,c^ilogŷ_N,c^i + ŷ_P,c^ilogŷ_P,c^i),
  
  where ŷ_N,c^i and ŷ_P,c^i 
  denote the prediction of 𝐱_t^i belonging to class c obtained by C_N(·) and C_P(·), respectively. 
  
  To sum up, we integrate adversarial dual classifiers training and cross-domain discriminative alignment together, and propose our overall objective function as:
  
  
      [ min_Gℒ_s + ℒ_em + λ_1  ℒ_dis + λ_2 ℒ_m,;                min_C_Nℒ_s  - λ_1 ℒ_dis, ]
  
  where λ_1 and λ_2 are hyper-parameters to balance the contribution of loss terms ℒ_dis, ℒ_m, respectively.
  
  Similar to existing adversarial networks training strategy, we freeze the generator G(·) to train classifiers, then freeze the parameters of the classifiers to update the generator G(·). It is noteworthy that only C_N(·) contains trainable parameters because C_P(·) only relies on the embedding features produced by the generator G(·). Meanwhile, inspired by <cit.>, in order to keep the performance of the networks on the source domain and detect target samples far from source domain support, we train our framework by three steps:
  
  
  Step A. We train the feature generator G(·) and classifier C_N(·) only on source domain 𝒟_s which is the same as supervised learning tasks. Due to C_P(·) does not have any trainable parameters, only parameters in G(·) and C_N(·) would be updated. Our model aims to detect target samples which are outside of source support from those which are close to support of source domain, keeping good ability and performance on classifying the source domain samples correctly is crucial and necessary. The optimization objective is defined as min_G,C_Nℒ_s.
  
  
  
  
  
  
  
  Step B. We need to assign unlabeled target domain samples pseudo labels by classifiers we already have. In our experiments, we explore the prediction results of C_P(·) to obtain pseudo labels of the target samples, which are experimentally proven to achieve better performance, and we will discuss it in the ablation analysis section. We fix the feature generator G(·) and update the classifier C_N(·) to maximize the distribution discrepancy between the classification results of C_N(·) and C_P(·) on the target domain, which can detect the target samples excluded by the source domain data support, and we obtain the training objective function as min_C_Nℒ_s - λ_1 ℒ_dis.
  
  
  
  
  
  
  
  
  Step C. We freeze the parameters of the classifier C_N(·) and update generator G(·) to minimize the distribution discrepancy between the predictions of C_N(·) and C_P(·) on the target domain, through which both C_N(·) and C_P(·) classifiers will have more similar and correct prediction on target domain samples. Furthermore, together with the discriminative cross-domain alignment, the generator G(·) 
  tends to couple the source and target domain closer but discriminative in the embedding feature space. The optimization objective is min_Gℒ_s + ℒ_em + λ_1 ℒ_dis + λ_2 ℒ_m.
  
  
  
  
  
  
  
   These three steps repeat once in each iteration in our experiments. The generator G(·) and classifier C_N(·) are initialized and pre-trained on source domain data. 
  
  
  
  § EXPERIMENTAL RESULTS
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
   §.§ Datasets & Experimental Setup
  
  
  Office-Home <cit.> consists of 15,500 images from 65 categories in 4 different domains: Artistic images (Ar), Clip Art (Cl), Product (Pr), and Real-World images (Rw). In total, by choosing any two domain as one task, we can build 12 cross-domain tasks to evaluate our proposed model.
  
  
  
  
  
  
  
  
  
  Office-31 contains 4,110 images of 3 domains: Amazon (A), Webcam (W), and DSLR (D) and each domain consists of 31 categories. We evaluate our method on 6 cross-domain tasks to testify the validation of our model. 
  
  
  
  
  
  
  
  
  
  
  Comparisons. We compare our proposed method with several state-of-the-art unsupervised domain adaptation models: Deep Adaptation Networks (DAN) <cit.>, Reverse Gradient (RevGrad) <cit.>, Joint Adaptation Networks (JAN) <cit.>, Self-Ensembling (SE) <cit.>, Multi-adversarial Domain Adaptation (MADA) <cit.>, Conditional Adversarial Domain Adaptation Networks (CDAN) <cit.>, Disentangled Semantic Representation (DSR) <cit.>, Domain-specific Whitening Transform & Min-Entropy Consensus (DWT-MEC) <cit.>, Minimum Centroid Shift (MCS) <cit.>, Adaptive Feature Norm Approach (AFN) <cit.>, Domain Symmetric Networks (SymNets) <cit.>, Bi-Directional Generation (BDG) <cit.>. All our experiments follow standard unsupervised domain adaptation protocols: all labeled source domain data and labels, as well as unlabeled target domain data are used for training. All comparisons are back-boned with ResNet-50 or using ResNet-50 features <cit.>. 
  
  
  
  Implementation Details. We implement our model with PyTorch and adopt ResNet-50<cit.> as the backbone. Specifically, a ResNet-50 network is pre-trained on ImageNet <cit.> and fine-tuned on the source domain, then applied to both source and target domain data to obtain the feature representation with dimension 2,048 without the last fully connected layer. G(·) is a two-layer fully-connected neural network, with hidden layer output as 1,024 followed by ReLU activation function, and the dropout probability retaining is 0.5. The output embedding features 𝐳_s/t dimension is 512. C_N(·) is a two-layer fully-connected neural network with 512 as the input and hidden layer dimension, the output dimension is the same as the number of categories in the whole label space C. Cosine similarity is accepted as the measurement metric Φ(·,·) in C_P(·).
  
  
  All parameters are updated with Adam optimizer <cit.> and the learning rate is set as 0.001 on Office-Home and Office-31 dataset. G(·) and C_N(·) are pre-trained and initialized on source domain data only with the learning rate as 0.1 for 2,000 iterations. We deploy SWD distance <cit.> as the discrepancy measurement function ℱ(·,·), and accept L-2 norm as ℋ(·) to evaluate the distribution divergence. λ_1 and λ_2 are fixed as 0.1 for all tasks. σ is set to be 0.03. For the prototypical classifier C_P(·), we initialize the class prototypes with the source domain features class centers μ_c^s=1/n_s^c∑_i=1^n_s^c𝐳_s^i, then update the prototypes with target domain category centroids representation μ_c^t = 1/n_t^c∑_j=1^n_t^c𝐳_t^j after obtaining the target domain samples pseudo labels ŷ_P,t iteratively till reaching convergence or the max step (which is set as 3), and return the last step C_P(·) prediction.
  
  All results reported in Tables <ref> and <ref> are the average of three random experimental results obtained by classifier C_P(·), and we will discuss the performances of C_N(·) and C_P(·) in the ablation study section.
  
  
  
  
  
  
  
  
  
  
   §.§ Comparison Results
  
  
  Table <ref> and Table <ref> report the classification results on target domain data of our proposed model and other comparative methods on Office-Home and Office-31 datasets respectively. All comparison results are from their original paper or quoted from <cit.>, as we adopt exactly the same settings. It is noteworthy that our proposed model outperforms state-of-the-art methods on all benchmark datasets in terms of average accuracy, and obtains the best or comparable performances to the state-of-the-art domain adaptation methods in most cases. Although the Office-Home dataset is more challenging than Office-31 due to more categories and samples, as well as significant distribution dissimilarity, our proposed model still improves the performance on most tasks, which demonstrates the efficiency and effectiveness of our proposed framework. 
  
  DAN and JAN are both MMD-based methods, which seek to eliminate the cross-domain distribution disparity and match the whole source and target domain to a shared domain-invariant feature space. DAN attempts to align feature representations from multiple layers through a multi-kernel variant of MMD. JAN aims to transfer joint distributions of multi-layers' activation of the networks across domains. With the help of additional domain adaptation terms (e.g., MMD), DAN and JAN lead to a significant performance boost over the source-only-trained model (i.e., ResNet-50) on most adaptation tasks.
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  RevGrad implements adversarial networks and applies gradient reversal layer to train a domain discriminator. CDAN and MADA both exploit multiplicative interactions between feature representations and category predictions as high-order features to promote the adversarial training. SE explores the use of self-ensembling for visual domain adaptation. DSR assumes that the data generation process is controlled by the semantic latent variables and domain latent variables independently, so employs a variational auto-encoder in order to reconstruct them. MCS designs a unified framework without accessing the source domain data and iteratively assigns pseudo labels to the target samples by an alternating minimization scheme. 
  
  DWT-MEC proposes domain alignment layers with feature whitening to match source and target domain distributions and leverages the unlabeled target data by Min-Entropy Consensus loss. AFN proposes a novel Adaptive Feature Norm approach to progressively adapting the feature norms of the two domains to a large range of values.
  
  SymNets exploits a novel adversarial classifiers networks and a two-level domain confusion scheme driving the learning of categories invariant intermediate features across domains. BDG bridges source and target domain through consistent classifiers interpolating two intermediate domains.
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
   §.§ Ablation Analysis
  
  
  In this section, we analyze the contribution and influence of several important terms and hyper-parameters sensitivity in our proposed model.
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  First, we discuss the influence of each component in our framework. By removing one of ℒ_dis, ℒ_m, and ℒ_em, while keeping other terms same as original AD^2CN, we obtain three variants AD^2CN w/o ℒ_dis, 
   AD^2CN w/o ℒ_m, and AD^2CN w/o ℒ_em.
  
  From Fig. <ref>, we notice that all three components contribute to improving the domain adaptation performance, while our proposed discriminative cross-domain alignment ℒ_m plays a more crucial role than others, i.e., discrepancy and entropy minimization loss.
  
  Secondly, we compare the performances of C_N(·) and C_P(·) while accepting ŷ_N,t or ŷ_P,t as target domain pseudo labels for ℒ_m. From the results in Fig. <ref>, we observe that results with ŷ_P,t as pseudo labels are better than the results with ŷ_N,t in most cases. Compared to C_N(·), which is trained on the source domain, C_P(·) is based on the target prototypes and keeps better performance even on the early training stage. Fig. <ref> shows several test samples that C_P(·) classifies correctly while C_N(·) cannot handle, which emphasizes the superiority of C_P(·).
  
  
  
  Thirdly, we discuss the necessity and effectiveness of two different types of classifiers in our framework. Table <ref> shows the selective target domain class-wise recognition accuracy on OfficeHome Ar → Cl case produced by the two classifiers C_N and C_P in our proposed model, as well as the number of samples in each class from the source and target domains. From the results we notice that for the categories having sufficient well labeled source samples as well as balanced target domain samples for training, C_N have better performance than C_P, while for other categories with imbalanced distribution across domains and insufficient labeled source samples for training, C_P always performs better than C_N. The observation proves that for imbalanced dataset, C_N and C_P have different speciality for different categories with various cross-domain distributions. More over, we show the comparison results of MCD <cit.>, SWD<cit.>, and our proposed model on Office-31 dataset in Table <ref>. MCD and SWD are two dual classifier adversarial frameworks for domain adaptation, but using two completely same structure neural networks classifiers. We also replace the C_N and C_P in our proposed model with two same structure neural networks classifiers and report the results as Ours(same). It is noteworthy that our proposed model achieves the best performance on most cases as well as the average accuracy compared to other same classifier structure methods, which proves the effectiveness and necessity of applying two distinct architecture classifiers.
  
  
  Fourthly, we visualize the t-SNE embeddings (Fig. <ref>) of feature representations generated by G(·) before and after the domain adaptation through our proposed model, in which each category is represented as a cluster and different colors denote the different domains. Before adaptation, the source and target domains are totally mismatched, while our method shows the promising ability to make inter-class separated and intra-class clustered tightly. 
  
  
  
  
  
  
  Finally, we analyze the sensitivity of λ_1 (Fig. <ref> (a))and λ_2 (Fig. <ref> (b)) by listing four tasks from Office-Home dataset (Ar → Cl, Cl → Pr, Pr → Rw, Rw → Ar). Specifically, we set the ranges of λ_1 and λ_2 from 0.001 to 0.2, and evaluate one by fixing the other one as 0.1. From the results, we notice the accuracy curves are almost flat and stable, which indicates our proposed model is not sensitive to the values of λ_1 nor λ_2.
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  § CONCLUSION
  
  
  We presented a novel Adversarial Dual Distinct Classifier Networks (AD^2CN) for unsupervised domain adaptation to align source and target domain distribution discrepancy as well as task-specific category boundaries. Specifically, we designed two different architecture classifiers to detect target samples excluded by the source domain support by aligning the task-specific decision boundaries obtained by the two classifiers. Meanwhile, a domain-invariant feature generator was proposed to embed source and target domain data to a shared feature space under the guidance of discriminative cross-domain alignment. We evaluated our proposed model on two cross-domain visual benchmarks and obtained better performance over state-of-the-art methods, proving the effectiveness of our method.
  
  ieee_fullname
  
  
  
  