Task-Discriminative Domain Alignment for Unsupervised Domain Adaptation
Behnam Gholami Pritish Sahu Minyoung Kim Vladimir Pavlovic








1]Behnam Gholami 

1]Pritish Sahu

2]Minyoung Kim

1,2]Vladimir Pavlovic

[1]Dept. of Computer Science, Rutgers University, NJ, USA
[2]Samsung AI Center, Cambridge, UK
[ ] , , 

























Task-Discriminative Domain Alignment for Unsupervised Domain Adaptation

    [
    December 30, 2023
========================================================================








Domain Adaptation (DA), the process of effectively adapting task models learned on one domain, the source, to other related but distinct domains, the targets, with no or minimal retraining, is typically accomplished using the process of source-to-target manifold alignment. However, this process often leads to unsatisfactory adaptation performance, in part because it ignores the task-specific structure of the data. In this paper, we improve the performance of DA by introducing a discriminative discrepancy measure which takes advantage of auxiliary information available in the source and the target domains to better align the source and target distributions. Specifically, we leverage the cohesive clustering structure within individual data manifolds, associated with different tasks, to improve the alignment. This structure is explicit in the source, where the task labels are available, but is implicit in the target, making the problem challenging. We address the challenge by devising a deep DA framework, which combines a new task-driven domain alignment discriminator with domain regularizers that encourage the shared features as task-specific and domain invariant, and prompt the task model to be data structure preserving, guiding its decision boundaries through the low density data regions. We validate our framework on standard benchmarks, including Digits (MNIST, USPS, SVHN, MNIST-M), PACS, and VisDA. Our results show that our proposed model consistently outperforms the state-of-the-art in unsupervised domain adaptation.







§ INTRODUCTION



Domain adaptation refers to the problem of leveraging labeled task data in a source domain to learn an accurate model of the same tasks in a target domain where the labels are unavailable or very scarce <cit.>. The problem becomes challenging in the presence of strong data distribution shifts across the two domains <cit.>, which lead to high generalization error when using models trained on the source for predicting on target samples. Domain adaptation techniques seek to address the distribution shift problem. The key idea is to bridge the gap between the source and target in a joint feature space so that a task classifier trained on labeled source data can be effectively transferred to the target <cit.>. In this regard, an important challenge is how to measure the discrepancy between the two domains. Many domain discrepancy measures have been proposed in previous DA studies, such as the moment matching-based methods <cit.>, and adversarial methods <cit.>. Moment matching-based methods use Maximum Mean Discrepancy (MMD) <cit.> to align the distributions by matching all their statistics. Inspired by Generative Adversarial Networks (GAN) <cit.>, adversarial divergences train a domain discriminator to discern the source from the target, while an encoder feature extractor is simultaneously learned to create features indistinguishable across the source and the target, confusing the discriminator.



Existing discrepancy approaches, reviewed in the next section, mainly focus on aligning domain-level feature distributions without considering category-level alignment. Thus, the alignment enforced by such discrepancy measures does not guarantee a good target performance as it ignores the cluster structure of the samples, aligned with their task labels. The assumption that the source features exhibit a well-defined cluster structure naturally transfers to the target: target features indicative of the same tasks as the source should manifest a similar cluster structure. In other words, when optimally aligned, the target features should amass around the source clusters such that the decision boundaries of the learned task classifiers do not induce partitioning of smooth clusters of target features. However, the aforementioned domain discrepancy measures only focus on global feature overlap, ignoring the finer task-aligned structure in the data. Consequently, they may inaccurately match the clusters and also cause the source features to form weakly separable clusters, as illustrated in <ref>, r2, r3.





To alleviate the limitations of existing discrepancy measures for domain adaptation, we introduce a task (e.g., classification)-specific adversarial discrepancy measure that extends the discriminator output over the source classes, in order to additionally incorporate task knowledge into the adversarial domain alignment. The new discrepancy measure helps the feature extractor (encoder) make discriminative source/target features by considering the decision boundary information. Consequently, source-target alignment not only takes into account the domain-level feature densities but also the category-conditioned clusters-of-features information to produce an improved overlap, evident in <ref>, part r4.

Motivated by the information-bottleneck principle <cit.>, whose goal is to improve generalization by ignoring irrelevant (domain-variant) distractors present in the original data features, we also introduce a source regularization loss by minimizing the information between the source samples and their features by encouraging the marginal distribution of the source features to be similar to a prior distribution (the standard normal distribution) to enforce the model to focus only on the most discriminative (label-variant) features, less prone to overfitting. Moreover, an additional target regularization term is imposed on the classifier, trained on the shared features of the source samples, to encourage it not to pass through high-density regions of the target data. Previous DA methods did not explicitly consider these desiderata. Our ablation study in <ref> empirically demonstrates the importance of the introduced objectives. We also empirically evaluate the advantages of our proposed method by demonstrating considerable improvements over the state-of-the-art methods on several standard domain adaptation benchmarks, including Digits, PACS and VisDA datasets.


































§ RELATED WORK



We summarize DA works most relevant to this paper. Several types of adversarial learning methods for unsupervised domain adaptation have been shown to match distributions of the features generated from source and target samples <cit.>.

The domain adversarial neural network (DANN) <cit.> first introduced a gradient reversal layer that reversed the gradients of the domain discriminator in order to encourage domain confusion.  Other recent proposals <cit.> have explored generative models such as GANs <cit.> to generate synthetic images for domain adaptation.  These approaches typically train two GANs on the source and target input data with tied parameters with the goal of translating images between the domains. Despite being visually compelling, such image-space models have only been shown to work on small images and limited domain shifts.

In order to circumvent the need to generate images, ADDA <cit.> recently proposed an adversarial framework for directly minimizing the distance between the source and target encoded representations (shared features). A discriminator and (target) encoder are iteratively optimized in a two-player game, where the goal of the discriminator is to distinguish the target features from the source features, with the goal of the encoder being to confuse the discriminator. The DupGAN <cit.> proposed a GAN-like model with duplex discriminators to restrict the latent representation to be domain invariant, with its category information preserved. Saito et al. <cit.> further introduce two classifiers as a discriminator to avoid ambiguous features near the class boundaries. By deploying two classifiers, the method therein employs the adversarial learning techniques to detect the disagreement across classifiers, such that the encoder is able to minimize this discrepancy on target samples. 


In addition to the adversarial distribution matching oriented algorithms, pseudo-labels or conditional entropy regularization are also adopted in literature <cit.>. Sener et al. <cit.> construct a k-NN graph of target points based on a predefined similarity graph. Pseudo-labels are assigned to target samples via their nearest source neighbors, which allows end-to end joint training of the adaptation loss. Saito et al. <cit.> employ the asymmetric tri-training, which leverages target samples labeled by the source-trained classifier to learn target discriminative features. Zhang et al. <cit.> iteratively select pseudo-labeled target samples based on their proposed criterion and retrain the model with a training set including pseudo-labeled samples. However, these methods based on pseudo-labeled target samples have a critical bottleneck where false pseudo-labels can mislead learning of target discriminative features, leading to degraded performance. 







§ METHOD





 §.§ Problem Formulation


Without loss of generality, we consider a multi-class (K-class) classification problem as the running task example. 
Consider the joint space of inputs and class labels, 𝒳×𝒴 where 
𝒴={1,…,K} for (K-way) classification. Suppose we have two domains on this joint space, source (S) and target (T), defined by unknown distributions P_S( x, y) and P_T( x, y), respectively.  We are given source-domain training examples with labels 𝒟_S = {( x^S_i, y^S_i)}_i=1^N_S and target data 𝒟_T = { x^T_i}_i=1^N_T with no labels.  We assume the shared set of class labels between the two domains. The goal is to assign the correct class labels {y^T_i} to target data points 𝒟_T.

To tackle the problem in the shared latent space framework, we introduce a shared encoder Q between the source and the target domains that maps a sample 𝐱 into a stochastic embedding[Please see Remark <ref> for the benefits of choosing a stochastic encoder over a deterministic one.] 𝐳∼ Q(𝐳|𝐱), and then apply a classifier h to map 𝐳 into the label space y ∼ h(y|𝐳) (h is trained
to classify samples drawn from the encoder distribution). Although one can consider domain-wise different encoders, more recent DA approaches tend to adopt a shared encoder, which can prevent domain-specific nuisance features from being learned, reducing potential overfitting issues. 

We define the stochastic encoder E as a conditional Gaussian distribution with diagonal covariance that has the form Q(𝐳|𝐱)=𝒩( z|f_μ(𝐱),f_Σ(𝐱)) where 𝐟_· is a deep network mapping the data point 𝐱 to the 2p-dimensional latent code, with the first p outputs from f_e encoding 𝐟_μ, and the remaining p outputs encoding 𝐟_Σ (in this work, we set p=256 for all the experiments). 

The classifier h outputs a K-dim probability vector of class memberships, modeled as a softmax form 𝐡(𝐳) = (𝐟_c(𝐳)),



where 𝐟_c(𝐳) is a deep network mapping the latents z to the logits of K classes.


The reason to choose a stochastic encoder over a deterministic one is two fold. First, it allows one to impose smoothness (local-Lipschtizness) constraint on the classifier h over target samples; see <ref> for more details. Second, adding continuous noise to the inputs of the discriminators has been shown to improve instability and vanishing gradients in adversarial optimization problems through smoothening the distribution of features <cit.>. Our stochastic encoder equipped with the reparametarization approach inherently provides such mechanism to feature distribution smoothness; see <ref> and <ref> for more details.   


The proposed domain adaptation method can be summarized by the objective function consisting of six terms:

    ℒ_Class + ℒ_Disc + ℒ_Teach + ℒ_Smooth + ℒ_Entropic + ℒ_Adv,

where ℒ_Class is the classification loss applied to 𝒟_S, ℒ_Disc is the domain discrepancy loss measuring the discrepancy between the source and target distribution, ℒ_Teach is the source-to-target teaching loss, which couples the source classifier with the target discriminator. The remaining losses,  ℒ_Smooth, ℒ_Entropic, ℒ_Adv will impose different regularization constraints on the model:  ℒ_Smooth will impose Lipschitz classifiers in the target space, ℒ_Entropic will strive to drive the classifier towards regions of low density in the same target space, while ℒ_Adv will impose regularization towards a reference density in the shared space 𝒵. We next discuss each of the above losses in more detail and then propose an algorithm to efficiently optimize the desired objective. 




  §.§.§ Source Classification Loss ℒ_Class


Having access to source labels, the stochastic mappings Q and h are trained on source samples to correctly predict the class label by minimizing the standard cross entropy loss,




    ℒ_Class(Q,h):=-𝔼_𝐱,y∼ P_S(𝐱,y)[ 𝔼_𝐳∼ Q(𝐳|𝐱)[ 𝐲^⊤log h(𝐳) ] ],

where 𝐲 is the K-D one-hot vector representing the label y.




  §.§.§ Domain Discrepancy Loss ℒ_Disc


Since the stochastic encoder Q is shared between the source and target samples, to make sure the source and the target  features are well aligned in the shared space and respect the cluster structure of the original samples, we propose a novel domain alignment loss, which will be optimized in adversarial manner.

Rather than using the standard adversarial approach to minimizing the alignment loss between the source and the target densities in the shared space 𝒵, i.e., finding the encoder Q which "fools" the best binary discriminator D trying to discern source from target samples, our approach is inspired by semi-supervised
GANs <cit.> where it has been found that incorporating task knowledge into the discriminator can
jointly improve classification performance and quality of images produced by the generator. 

We incorporate task knowledge by replacing what would be a binary discriminator with a (K+1)-way multi-class discriminator y' = D(𝐳) = (𝐟_d(𝐳)).

The first K classes indicate that a sample 𝐳 belongs to the source domain and belongs to a specific classes in 𝒴, while the last (K+1)-th class "t" indicates 𝐳 belongs to the target domain.

 Since we have the class label for the source samples, the discriminator is trained to classify source features correctly, hence creating crisp source clusters in the feature space. On the other hand, the new discriminator seeks to distinguish the samples from the target domain from those of the source by assigning them to the (K+1)-th, "target" class.
 







    ℒ_Disc(Q,D) := 
        -𝔼_𝐱∼ P_T(𝐱)[ 𝔼_𝐳∼ Q(𝐳|𝐱)[ [0,1]^⊤log D(𝐳) ] ] 
    
        - 𝔼_𝐱,y∼ P_S(𝐱,y)[ 𝔼_𝐳∼ Q(𝐳|𝐱)[ [𝐲,0]^⊤log D(𝐳) ]],






where [0,1] is a one-hot vector indicating a point from the target domain and [𝐲,0] stands for a point from the source domain, labeled according to class label 𝐲. 





































  §.§.§ Teacher Target-Source Loss ℒ_Teach


Here, we seek the encoder Q to generate a feature representative of one of the first K task-specific classes for target samples preserving their cluster structure and aligning them to the source clusters in the feature space. However, the target data points are unlabeled, and the encoder will not have the chance to enforce the desired clustering structure of the target points, where points within a cluster would have the same predicted label. To "teach" the encoder, we ask the classifier h(·) to provide pseudo soft labels for the target points to our new discriminator using the following loss:

    ℒ_Teach(Q,D,h):= 
    
        -𝔼_𝐱∼ P_T(𝐱)[ 𝔼_𝐳∼ Q(𝐳|𝐱)[ [h(𝐳),0]^⊤log D(𝐳)] ].

Intuitively, the encoder tries to fool the discriminator by assigning one of the first K classes to target features, leveraging on the output of the classifier h (augmented with 0 for the K+1-th dimension) as pseudo-labels for target features.


The proposed task-specific domain discriminator can be used to improve any domain adaptation method that has an adversarial domain alignment component. Indeed, we observe (see <ref>) that the proposed discriminator significantly improves upon the standard binary discriminator.





  §.§.§ Source Domain Regularization Loss



One of the standard goals in representation learning is to find an encoding of the data point 𝐱 that is maximally expressive about its label y while being maximally compressive about 𝐱—finding a representation 𝐳 which ignores as many details of 𝐱 as possible. This is specifically useful for domain adaptation where we require a representation to be domain invariant. Essentially, we want 𝐳 to act like a minimal sufficient statistic of 𝐱 for predicting y in order to generalize better for samples form unseen domains. To do so, we introduce a regularizer that acts on the aggregated posterior of the shared features of the source samples Q_z(𝐳) = 𝔼_𝐱∼ P_S(𝐱) [ Q(𝐳| 𝐱)]. The regularizer encourages 𝐳 to be less informative about 𝐱 in the form of mutual information by matching the aggregated posterior of the shared features with a factorized prior distribution P_z(𝐳)[In this work, we consider P_z(𝐳) = 𝒩(0,𝐈)], which in turn constrains the implicit capacity of 𝐳 and encourages it be factorized:

    𝒟[P_z(𝐳)||Q_z(𝐳)],

where 𝒟(·||·) is an arbitrary distribution divergence measure. 

As the proxy for this divergence, we define an auxiliary loss which will be adversarially optimized. We introduce an a binary discriminator F in the latent space trying to separate true points sampled from P_z and fake ones sampled from Q_z. The encoder Q ensures the aggregated posterior distribution Q_z can fool the binary discriminator into thinking that the source features comes from the distribution P_z:




    ℒ_Adv(Q,F) = -𝔼_x∼ P_S(𝐱)[ 𝔼_𝐳∼ Q(𝐳|𝐱)[ log F(𝐳) ]] 
    
        -𝔼_𝐳∼ P(𝐳)[ log (1-F(𝐳)) ].


We empirically observed that imposing such regularization on target samples could be harmful to performance. We conjecture this is due to the lack of true class labels for the target samples, without which the encoder would not preserve the label information of the features, leading to unstructured target points in feature space. 













  §.§.§ Target Domain Regularization Losses


In order to incorporate the target domain information into the model, we apply the cluster assumption, which states that the target data points 𝒟_T contains clusters and that points in the same cluster have homogeneous class labels. If the cluster assumption holds, the optimal decision boundaries should occur far away from data-dense regions in the feature space z. We achieve this by defining an entropic loss,

    ℒ_Entropic(h,Q):= 
     
         -𝔼_𝐱∼ P_T(𝐱)[ 𝔼_𝐳∼ Q(𝐳|𝐱)[ h(𝐳)^⊤log h(𝐳) ] ].

Intuitively, minimizing the conditional entropy forces the classifier to be confident on the unlabeled target data, thus driving the classifier’s decision boundaries away from the target data. In practice, the conditional entropy must be empirically estimated using the available data.
 
However, Grandvale <cit.> suggested this approximation can be very poor if h is not locally-Lipschitz smooth. Without the smoothness constraint, the classifier could abruptly change its prediction in the neighborhood of training samples, allowing decision boundaries close to the training samples even when the empirical conditional entropy is minimized. To prevent this, we take advantage of our stochastic encoder and propose to explicitly incorporate the locally-Lipschitz constraint in the objective function,

    ℒ_Smooth(h,Q):= 
    𝔼_𝐱∼ P_T(𝐱)[ 𝔼_𝐳_1,𝐳_2∼ Q(𝐳| x)||h(𝐳_1) - h(𝐳_2)||_1 ],

with ·_1 the L_1 norm. Intuitively, we enforce classifier consistency over proximal features of any target point 𝐱. 


We empirically observed that having such constraints for source features would not improve performance. This is because access to the source labels and forcing the classifier to assign each source feature to its own class would already fulfill the smoothness and entropy constraints on the classifier for the source samples. 





 §.§ Model Learning and Loss Optimization





Our goal, as outlined in <ref>, is to train the task-specific discriminator D, binary discriminator F, classifier h, and encoder Q to facilitate learning of the cross-domain classifier h. By approximating the expectations with the sample averages, using the stochastic gradient Descent (SGD), and the reparametarization approach <cit.>, we solve the optimization task in the following four subtasks. The overall algorithm is available in the Supplementary Material (SM).





  §.§.§ Optimizing the encoder Q




    Q^* = min_Q  ℒ_Class(Q,h^*) + ℒ_Disc(Q,D^*,h^*) 
    
      + λ_Q[ℒ_Adv(Q,F^*)],






where λ_Q is a weighting factor. Intuitively, The first term in <ref> encourages Q to produce discriminative features for the labeled source samples to be correctly classified by the classifier h. The second term simulates the adversarial training by encouraging Q to fool the task-specific discriminator D by pushing the target features toward the source features, leveraging the soft pseudo-labels provided by the classifier. Through the last term, the encoder seeks to fool the binary discriminator F into treating the source features as if they come from the fully-factorized P(𝐳) to produce domain-invariant source features. 




  §.§.§ Optimizing the classifier h



    h^* = min_h λ_h[ℒ_Class(Q^*,h)] 
    
        + λ'_h[ℒ_Entropic(Q^*,h)
        + ℒ_Smooth(Q^*,h)],







where λ_h and λ'_h are the trade-off factors. Intuitively, we enforce the classifier h to correctly predict the class labels of the source samples by the first term in <ref>. We use the second term to minimize the entropy of h for the target samples, reducing the effects of "confusing" labels of target samples. The last term guides the classifier to be locally consistent, shifting the decision boundaries away from target data-dense regions in the feature space.    




  §.§.§ Optimizing the task-specific discriminator D






    D^* = min_D ℒ_Disc(Q^*,D).

The loss in <ref> prompts D to shape its decision boundary to separate the source features (according to their class label) and target features from each other.




  §.§.§ Optimizing the binary discriminator F



    F^* = min_F ℒ_Adv(Q^*,F).





Intuitively, the loss in <ref> encourages F to separate the source features from the features generated from the fully-factorized distribution P_z(𝐳) by assigning label 1 and 0 to the source feature samples and P_z(𝐳) samples, respectively.




 §.§ Target Class Label Prediction



After model training, to determine the target class-label y_t
of a given target domain instance 𝐱_t, we first compute the distribution of y_t given 𝐱_t by integrating out the shared feature 𝐳_t. Then, we select the most likely label as

    ŷ_t = max_y_t∈{1,...,K}  P(y_t|𝐱_t),

where P(y_t|𝐱_t) can be computed as

    P(y_t = k|𝐳_t)=𝔼_𝐳∼ Q( 𝐳|𝐱_t)=𝒩(f_e^μ(𝐱_t),f_e^Σ(𝐱_t))[ h_k(𝐳)],

where h_k(.) is the k-th entry of the classifier output.
Since the above expression cannot be computed in a closed form, we approximate it with its mean value. Using this approximation, we compute y_t as:

    ŷ_t = max_k∈{1,...,K}  h_k(𝐳_t),   𝐳_t = f_e^μ(𝐱_t).
 

We empirically observed that estimating the expectation in <ref> with Gibbs sampling from the posterior Q(𝐳|𝐱_t) instead of its mean would not boost the performance. We conjecture this is due to the smoothness constraint we impose on the classifier through <ref>, enforcing consistency over proximal target samples drawn from Q(𝐳|𝐱).  




§ EXPERIMENTAL RESULTS


We compare our proposed method with state-of-the-art on three benchmark tasks. The Digit datasets embody the digit cross-domain classification task across four datasets: MNIST, MNIST-M, SVHN, USPS, which consist of K = 10 digit classes (0-9). We also evaluated our method on VisDA object classification dataset <cit.> with more than 280K images across twelve categories. Finally, we report performance on PACS <cit.>, a recently proposed benchmark which is especially interesting due to the significant domain shift between different domains. It contains images of seven categories extracted from four different domains:Photo (P), Art paintings (A), Cartoon (C), and Sketch (S). The details of the datasets are available in SM.  <ref> illustrates image samples from different datasets and domains. We evaluate the performance of all methods with the classification accuracy metric. We used ADAM <cit.> for training; the learning rate was set to 0.0002 and momentums to 0.5 and 0.999. Batch size was set to 16 for each domain, and the input images were mean-centered/rescaled to [-1,1]. All the used architectures replicate those of state-of-the-art methods, detailed in SM. We followed the protocol of unsupervised domain adaptation and did not use validation set to tune the hyper-parameters λ_Q,λ_h,λ'_h. Full hyper-parameter details for each experiment can be found in SM. We compare the proposed method with several related methods, including CORAL <cit.>, DANN <cit.>, ADDA <cit.>, DTN <cit.>, UNIT <cit.>, PixelDA <cit.>, DIAL <cit.>, DLD <cit.>, DSN <cit.>, and MCDA <cit.> on digit classification task (Digit datasets), and the object recognition task (VisDA and PACS datasets).





 §.§ Results On Digits Recognition


In this evaluation, we follow the same protocol across all methods. Specifically, 
we use the network structure similar to UNIT <cit.>. See SM for more details.

We show the accuracy of different methods (averaged over five different runs) in <ref>. The proposed method outperformed the competing methods in five out of six  settings, confirming consistently and significantly better generalization of our model over target data. 

The higher performance of the proposed model compared to other methods is mainly attributed to the proposed task-specific alignment method, which not only encourages the source features to be well-separated, according to their class label, but also aligns the target to source features in a cluster-wise manner, "matching" the source and target clusters. This is in contrast to the standard domain-wise alignment, which ignores the source/target inherent cluster structure. This superiority also benefits from the proposed source and target domain regularizers, which improve the source feature domain-invariance and the classifier's robustness respectively. See <ref> for more details. 







































 §.§ Results on Object Recognition


We also evaluate our method on two object recognition benchmark datasets VisDA <cit.> and PACS <cit.>. We follow MCDA <cit.>, and use ResNet101 <cit.> as the backbone network which was pretrained on ImageNet dataset, and then finetune the parameters of ResNet101 with the source only VisDA dataset according to the procedure described
in <cit.>. For the PACS dataset, we also follow the experimental protocol in <cit.>, using ResNet18 <cit.> pretrained on ImageNet dataset, and training our model considering 3 domains as sources and the remaining as target, using all the images of each domain. 
For these experiments, we set the learning rate of resnets to 10^-9. We choose this small learning
rate for ResNet parameters since the domain shift for
both VisDA and PACS are significant, the training procedure benefits from a mild parameter updates back-propagated from the
loss.

Results for this experiment are summarized in <ref> & <ref>. We observe that our model achieved, on average, the best performance compared to other competing methods for both datasets. The higher performance of our method is mainly attributed to incorporating the category-level information into the domain alignment through the proposed task-specific discriminator, which is beneficial to boost the discriminability of the source/target features.








 §.§ Analysis of the task-specific discriminator


To measure how effective the new task-specific discriminator is, we conducted an experiment to compare the task-specific discriminator with the standard adversarial discriminator (training a logistic
function on the discriminator by assigning labels 0 and 1 to the source and target domains respectively
and training the encoder with inverted labels). The results are shown in <ref>. As is evident from the figure, there is a substantial increase in accuracy over all adaptation scenarios on switching from the standard adversarial discriminator to our task-specific discriminator. The superiority of the performance is mainly due to explicitly accounting for task knowledge
in the proposed discriminator during adversarial training that encourages the discriminativity of the source/target samples in the feature space.

We further visualize the distribution of the learnt shared features to investigate the effect of task-specific discriminator (Task-d) and its comparison to adversarial discriminator (Adv-d). We use t-SNE <cit.> on SVHN to MNIST adaptation to visualize shared feature representations from two domains. <ref> shows shared features from source (SVHN) and target (MNIST) before adaptation (vis_original,vis_class), after adaptation with Adv-d (vis_binary,vis_binary_class), and after adaptation with Task-d (vis_ours, vis_ours_class).

While a significant distribution gap is present between non-adapted features across domains (vis_original), the domain discrepancy is significantly reduced in the feature space for both Adv-d (vis_binary) and Task-d (vis_ours). On the other hand, adaptation with Task-d led to pure and well-separated clusters in feature space compared to the adaptation with Adv-d, and leads to superior class separability. As supported by the quantitative results in <ref>, this implies that enforcing clustering in addition to domain-invariant embedding was essential for reducing the classification error. This is depicted in vis_ours_class, where the points in the shared space are grouped into class-specific subgroups; color indicates the class label. This is in contrast to <ref>, where the features show less class-specificity.




















 §.§ Ablation Studies


We performed an ablation study for our unsupervised domain adaptation approach on Digit dataset. Specifically, we considered training without source regularization, denoted as Ours (w/o-s), training without target regularization, Ours (w/0-t), and training by excluding both the source and the target regularization, Ours (w/o-st).

The results are shown in <ref>. As can be seen, removing one or more of the objectives results in noticeable performance degradation. The more parts are removed, the worse the performance is.  More precisely, disabling the source regularizer results in an average ≈ 3.5% drop in performance. That demonstrates that the source regularizer can improve the generalization over target samples by encouraging the source features to be domain-invariant, less informative about the identity of either of the domains.  Immobilizing the target regularizer leads to ≈ 2.0% average drop in performance. These results strongly indicate that it is beneficial to make use of the information from unlabeled target data the during classifier learning process, which further strengthens the feature discriminability in the target domain.  Finally, the average performance drop that stems from disabling both the source and the target regularizer is ≈ 5.5%.  This suggests that the two components operate in harmony with each other, forming an effective solution for domain adaptation.




§ CONCLUSION



We proposed a method to boosts the unsupervised domain adaptation by explicitly accounting for task knowledge in the cross-domain alignment discriminator, while simultaneously exploiting the agglomerate structure of the unlabeled target data using important regularization constraints. Our experiments demonstrate the proposed model achieves state-of-the-art performance across several domain adaptation benchmarks.

ieee



