Transferable Joint Attribute-Identity Deep Learning for Unsupervised Person Re-Identification
Jingya Wang Xiatian Zhu Shaogang Gong Wei Li




























Transferable Joint Attribute-Identity Deep Learning for 
 Unsupervised Person Re-Identification
    Jingya Wang^1 
	        Xiatian Zhu^2 
	        Shaogang Gong^1 
	        Wei Li^1 

Queen Mary University of London^1         

Vision Semantics Ltd.^2

{jingya.wang, s.gong, w.li}@qmul.ac.uk 
    eddy@visionsemantics.com

    .
===================================================================================================================================================================================================================================






Most existing person re-identification (re-id) methods require supervised model learning 
from a separate large set of pairwise labelled training data for every single camera pair.
This significantly limits their scalability and usability in real-world 
large scale deployments with the need for performing re-id across many camera views. 

To address this scalability problem, we develop a novel 
deep learning method for transferring the labelled information of
an existing dataset to a new unseen (unlabelled) target domain for
person re-id without any supervised learning in the target domain.

Specifically, we introduce an Transferable Joint Attribute-Identity Deep Learning
(TJ-AIDL) for simultaneously learning an attribute-semantic and identity-discriminative
feature representation space transferrable to 
any new (unseen) target domain for re-id tasks 
without the need for collecting new labelled training data from the target
domain (i.e. unsupervised learning in the target domain).

Extensive comparative evaluations validate the superiority of this new TJ-AIDL model 
for unsupervised person re-id over a wide range of state-of-the-art methods on four challenging benchmarks including
VIPeR, PRID, Market-1501, and DukeMTMC-ReID. 
	









§ INTRODUCTION



Person re-identification (re-id) aims at matching people across non-overlapping camera views
distributed at distinct locations.

Most existing re-id studies follow the supervised learning paradigm
such as 
optimising pairwise matching distance metrics <cit.>
or deep learning methods
<cit.>.

They assume the availability of a large number of manually
labelled matching pairs for each pair of 
camera views for learning a feature representation or a matching
distance function optimised for that camera pair.

However, this leads to a poor scalability in 
practical re-id deployments, because
such scale manual labelling 
is not only prohibitively expensive to collect in the real-world as there are
a quadratic number of camera pairs, but also implausible in many cases,
e.g. there may not exist sufficient
training people reappearing in every pair of camera views.

This scalability limitation severely 
reduces the usability of existing supervised re-id methods.

One generic solution to large scale re-id in real-world deployment is
designing unsupervised models.
While a few unsupervised methods
have been developed
<cit.>,
they typically offer weaker re-id performances when compared to the supervised counterparts.
This makes them less useful in practice.
One main reason is that without labelled data across views,
unsupervised methods lack the necessary knowledge on how visual
appearance of identical objects changes cross-views
due to different view angles, background and illumination. 


Another solution is to exploit simultaneously 
(1) unlabelled data from a target domain 

and 
(2) existing labelled datasets from some training source domains. 

Specifically, the idea is to learn a feature representation that contains some
view-invariant information about people appearance learned from labelled source data,
transfer and adapt it to a target domain by using only unlabelled
target data for re-id matching in the target domain. 

As the target dataset has no label, 
this is regarded as an unsupervised learning problem. 




There are a few studies on exploiting unlabelled target data 
for unsupervised re-id modelling using either identity or attribute label, or both
from source datasets
<cit.>.

However, they generally offer weaker re-id performance 
due to either domain sensitive hand-crafted features
or a lack of an effective knowledge transfer learning algorithm
between attribute and identity discriminative features.

It is very challenging to address
this cross-domain and multi-task (between attribute and identity)
transfer learning problem in a principled way due to three co-occurring uncertainties:
(1) Source and target domains have unknown camera viewing conditions;
(2) The identity/class population between source and target
domains are non-overlapping therefore presents a more challenging
open-set recognition problem, as compared to the closed-set assumption made
by most existing transfer learning models <cit.>;
(3) Joint exploitation of attribute and identity labels
suffers from the heteroscedasticity
(a mixture of different knowledge granularity and characteristics) 
learning problem <cit.>.



In this work, we consider unsupervised person re-id
by sharing the source domain knowledge through attributes learned from
labelled source data and transfering such knowledge to unlabelled target data by a
joint attribute-identity transfer learning across domains.
We make three contributions:
(I) 
We formulate a novel idea of heterogeneous multi-task joint deep learning 
of attribute and identity discrimination for unsupervised person re-id.



To our best knowledge, this is the first attempt at joint deep learning
of auxiliary attribute and identity labels 
for solving the unsupervised person re-id problem cross-domains. 

(II)
We propose a Transferable Joint Attribute-Identity Deep Learning (TJ-AIDL)
to simultaneously learn global identity 
and local attribute information from labelled source domain person images
through an Identity Inferred Attribute (IIA) space
for maximising the joint learning effectiveness
between identity and attribute.

This IIA is designed specially to address the
notorious heteroscedasticity challenge from which
the common space multi-task joint learning often suffers.

Importantly, the IIA interacts concurrently with both the attribute and identity learning tasks
inter-dependently without breaking 
the end-to-end model learning process.

(III)
We introduce an attribute consistency scheme
for performing TJ-AIDL model unsupervised adaptation on the unlabelled target data
to further enhance its discriminative compatibility towards
each target domain re-id task at hand.

Extensive evaluations demonstrate the superiority of the
proposed TJ-AIDL model over a wide range of state-of-the-art
re-id models on four challenging benchmarks
VIPeR <cit.>,
PRID <cit.>,
Market-1501 <cit.>,
and DukeMTMC-ReID <cit.>.










§ RELATED WORK


Person Re-ID
Most existing re-id models are based on supervised learning
for every camera pair
on a separate set of labelled training data
<cit.>.
They suffer from
poor scalability in 
realistic re-id deployments
where no such a large training set is available
for each single camera pair.

To solve this scalability issue,
unsupervised methods based on hand-crafted features
<cit.> can be chosen for deployment.
However, they usually yield much weaker performance
than supervised models therefore practically not very useful.

While a balance between scalability and matching accuracy
can be achieved by semi-supervised learning,
existing methods <cit.> still demand a fairly large set of pairwise labels 
which is again not scalable.



Recently, unsupervised re-id by cross-domain transfer learning has been developed 
to exploit labelled data from source datasets by extracting
transferable identity-discriminative information to an unlabelled target dataset <cit.>.
However, these methods have a few limitations that restrict their generalisation:
(1) Relying on hand-crafted features 
without the deep learning capability of automatically 
learning stronger representations from training data
<cit.>;
(2) Using a pre-learned deep model on labelled source data
but lacking an effective domain adaptation mechanism <cit.>;
(3) Independently exploiting identity and attribute label supervision
in model learning therefore ignoring their interaction and compatibility
<cit.>.
Data synthesis <cit.> has also been
proposed as a solution for addressing limited data, although it
suffers from undesirable person appearance distortion and restricted source selection. 


The proposed TJ-AIDL method addresses these limitations of existing methods
in a unified deep joint learning model.
Moreover, our method goes beyond the common multi-task joint learning
design by introducing a more transferable mechanism for
discriminatively optimising both attribute and identity learning in a shared end-to-end process.

Our experiments show that the proposed method significantly 
outperforms existing models even by using less supervision in the source domain.



Attribute for Re-ID


Visual semantic attributes <cit.> have been exploited as a mid-level feature representation for cross-view re-id 
<cit.>. 
However, such semantic coefficient representations are less
powerfull for identity discrimination than conventional feature vectors. 
The reasons are: 
(1) Attribute coefficient representations are usually of low dimensions 
(tens vs. thousands for typical low-level feature representations)
<cit.>; 
(2) Consistently predicting individually all the attributes is a
difficult task when the labelled training data is sparse and person
images have low quality as mostly in person re-id datasets, that is,
inter-attribute discrimination can be weak on typical person re-id images.



To overcome these problems, we explore attributes in our TJ-AIDL model
by introducing a mechanism to extract identity discriminative
attribute information through co-learning both attribute and identity labelled data jointly. 

Moreover, we uniquely employ the attribute space for unsupervised domain adaptation.














































§ A JOINT ATTRIBUTE-IDENTITY SPACE


Problem Definition
For person re-id by attribute (semantic) based unsupervised domain adaptation,
we have a supervised source dataset (domain) 
{(I_i^s, y_i^s, a_i^s)}_i=1^N^s consisting of N^s person 
bounding box images I^s, 
the corresponding identity y^s ∈𝒴={1,⋯,N^s_id} 
(i.e. a total N_id^s different persons), and 
identity-level binary attribute a_s ∈ℛ^m×1 
(i.e. a total m different attributes)
labels.

We also assume a set {I_i^t}_i=1^N^t of N^t unlabelled target training data,
which can be used for model domain adaptation.

The objective is to develop an unsupervised domain adaptation approach 
to learning the optimal feature representation
by transferring the supervised identity and attribute knowledge of the source domain 
to person re-id in a target domain with only unlabelled data of
entirely a different pool of identity classes.

Note that: 
(1) Unlike identity label, 
attribute detection is a multi-label recognition problem since
the m attribute categories co-exist in every single person
image.
(2)
These two types of label supervision
lie at quite different levels: Most attributes are localised to image
regions, even though the location information is not
provided in the annotation; While
person identity labels are at the holistic image-level.

It is a non-trivial learning task since this is not only a multi-label learning problem – joint
learning of mutually correlated attribute labels, 
but also a
heterogeneous multi-task joint learning problem – inter-dependently
learning a person re-id representation space by joint holistic identity and 
local attribute supervision.



In this work, we present a novel Transferable Joint Attribute-Identity Deep Learning (TJ-AIDL) approach to 
establishing an identity-discriminative and attribute-sensitive (i.e. dually-semantic) feature representation
space optimal for person re-id on the labelled target domain without any identity and attribute labels provided.


We avoid simply combining re-id and attribute feature vectors in deep model design to gain their 
complementary advantages, which may suffer from the heteroscedasticity 

problem <cit.>
and finally results in sub-optimal results.

Instead, we assign them into two separate branches for simultaneously learning
individual discriminative features subject to the corresponding 
label supervision concurrently.

Importantly, we design a progressive knowledge fusion mechanism by 
introducing an Identity Inferred Attribute (IIA) regularisation space

for more smoothly

transferring the global identity information into 
the local attribute feature representation space.

It is also the proposed IIA space that provides an opportunity
that allows for adapting the learned model to the target domain
where no identity and attribute labels are available. 







As such, the proposed TJ-AIDL largely addresses the joint learning challenges of
heterogeneous identity and attribute label information sources in a shared representational space
in a more challenging cross-domain context.
















 §.§ Transferable Joint Deep Learning


Model Overview 
We consider a multi-branch network architecture 
for our heterogeneously supervised multi-task learning.
The rational of this multi-branch composition 
is to maintain a sufficient independence of each supervision learning tasks
for avoiding their potentially negative mutual influence due to their semantic 
discrepancy. 


An overview of the proposed Transferable Joint  Attribute-Identity Deep Learning (TJ-AIDL) method
is depicted in Fig. <ref>. 
The TJ-AIDL contains two branches:
(1) Identity Branch: which aims to extract the re-id sensitive information from
the available identity class labels in the source domain (Figure <ref>(a)).
(2) Attribute Branch:
which aims to extract the semantic knowledge from the attribute labels 
(also from the source domain) (Figure <ref>(b)).

To establish a channel for knowledge fusion,
we introduce the Identity Inferred Attribute  (IIA) space (Figure <ref>(c))

designed for transferring the re-id discriminative information from the Identity Branch 
to the Attribute Branch where two-source information is synergistically 
integrated in a smoother manner.

That is, once the TJ-AIDL is trained, 
the feature representations extracted from the Attribute Branch can be 
directly exploited for re-id deployment.

For unsupervised person re-id by cross-domain knowledge transfer and target data adaptation, 
we conduct the model training of our proposed 
TJ-AIDL in two steps:
(I) Attribute-Identity Transferable Joint Learning:
This is supervised by the source labelled training data;
(II) Unsupervised Target Domain Adaptation:
This is performed on the target unlabelled training data.
We describe more details for each component of our TJ-AIDL
in two training steps.











  §.§.§ Attribute-Identity Transferable Joint Learning



Identity and Attribute Branches
For building an efficient yet strong deep re-id model,
we choose the lightweight MobileNet as the CNN architecture[
	This selection is independent of our model design
	and others can be readily applied, e.g.

	ResNet <cit.>, Inception <cit.> and VggNet <cit.>.]
for both identity and attribute branches.



For training the identity branch (Fig. <ref>(a)), we use the softmax Cross Entropy 
loss function defined as:






    L_id=-1/n_bs∑_i=1^n_bs
    log( p_id(I_i^s, y_i^s) )

where p_id(I_i^s, y_i^s) specifies the predicted probability on the groundtruth 
class y_i^s of 
I_i^s,
and n_bs denotes the batch size.

Given that the attribute branch (Fig. <ref>(b)) is a multi-label classification learning task,
we instead use the Sigmoid Cross Entropy loss function to generate the training signal
by considering all m attribute classes:

    L_att=-1/n_bs∑_i=1^n_bs∑_j=1^m( 
    a_i,jlog( p_att(I_i,j) ) + 
    
    (1-a_i,j) log( 1 - p_att(I_i,j) )
    )

where a_i,j and p_att(I_i,j) 
define the groundtruth label
and the predicted classification probability on the j-th attribute class
of the training image I_i,
i.e. a_i = [a_i,1,⋯,a_i,m]
and p_att,i = [p_att(I_i,1),⋯,p_att(I_i,m)].

By independently training the two branches using the above designs,
we only allows for optimising their respective features without
exploiting their complementary effect for maximising
the compatibility.
A common approach is to build a multi-task joint learning network
which directly subjects a shared feature representation to both 
identity loss (Eq. (<ref>))
and attribute loss (Eq. (<ref>)) 
concurrently in model training.
Instead, we present an alternative progressive scheme for 
more effective multi-source knowledge fusion 
as described below (see evaluations
in Sec. <ref>).
  



























Identity Inferred Attribute Space
We introduce an intermediate 
Identity Inferred Attribute (IIA) Space 
for achieving the knowledge fusion learning on
attribute and identity labels in a softer manner
(Fig. <ref>(c)).
The IIA space is jointly learned with the two branches 
while being exploited to perform information transfer and fusion from the identity branch
to the attribute branch simultaneously.
This scheme allows for both consistent and cumulative knowledge fusion in the whole training course.




More specifically, we build the IIA space in the encoder-decoder (auto-encoder) framework
due to that: (1) It has a strong capability of capturing the most important information 
of a given target task (represented by the input data)
via a concise feature vector representation;
(2) More importantly, such a concise feature representation
facilitates the inter-task information transfer whilst still
preserving sufficient updating freedom space to every individual learning task
<cit.>.

We call this sub-model IIA encoder-decoder.

In our context, we want to extract and compress essential identity information
into the IIA space for facilitating fusion.
We therefore exploit the identity features (Fig. <ref>(a))
as the input of IIA encoder and also
the groundtruth of IIA decoder 
(i.e. reconstruction unsupervised learning).
Once the input is given, this model itself can be
learned based on the reconstruction loss (Mean Square Error (MSE)): 

    L_rec  = x_id - f_IIA(x_id) ^2

where x_id represents the identity feature of a training image
and f_IIA() the mapping function of IIA encoder-decoder.
By this unsupervised learning manner, 
we are able to obtain a latent feature embedding e_IIA 
with important identity information encoded. 


To transfer the identity information across branches, 
we need a corresponding low dimensional matchable space in the attribute counterpart,
which however is not available.



To address the above problem, we propose to align the IIA embedding e_IIA 
with the prediction distribution over all m attribute classes,
in spirit of knowledge distillation <cit.>.

As such, we naturally set m as the dimension of e_IIA for easing alignment and 
cross-branch knowledge transfer without the need for an additional transformation.


More formally, we conduct the identity knowledge transfer via imposing an
MSE based identity transfer loss:

    L_ID-transfer  = e_IIA - p̃_att^2

where p̃_att is logits from the attribute branch. 
Considering that the e_IIA is derived in an unsupervised manner
which may be over further way from the attribute prediction counterpart and hence 
giving a harder alignment task,
we add similarly a sigmoid Cross Entropy loss to the learning of e_IIA
by exploiting it as a pseudo attribute prediction, as












    L_attr, IIA=-1/n_bs∑_i=1^n_bs∑_j=1^m( 
    a_i,jlog( p_IIA(I_i,j) ) + 
    
    (1-a_i,j) log( 1 - p_IIA(I_i,j) )
    )

where p_IIA(I_i,j) is the probability predicted based on e_IIA by the sigmoid function.


Finally, we formulate the overall IIA loss function by incorporating the above components
by weighted summation as:

    L_IIA = L_attr, IIA + λ_1L_rec + λ_2L_ID-transfer

where λ_1 and λ_2 are scale normalisation parameters to 
ensure all three loss quantities are of a similar scale in value.




Impact of IIA on Identity and Attribute Branches
The introduction of IIA imposes different influence on the two branches in model training.
Since IIA is established on the identity features,
no change is imposed into the learning of this branch.
For the attribute branch, however, an additional learning constraint 
is created for identity knowledge transfer.
We therefore reformulate its supervised learning loss function
by incorporating Eq. (<ref>) as:

    L_att-total=L_att + 
    λ _2 L_ID-transfer





Remarks
The IIA component aims at creating an interactive learning mechanism
between the identity and attribute branches in a more transferable way.
This significantly differs from the straightforward joint learning approach
which suffers from the underlying multi-source information incompatible problem.

We summarise the main information flow in model joint training:
(1) The identity branch learns to extract identity discriminative information;
(2) The IIA component then transfers the identity information to the attribute branch;
(3) The attribute branch learns to extract attribute discriminative knowledge whilst
simultaneously incorporating/fusing identity sensitive information.  


However, the TJ-AIDL model learned on the labelled source data is still not optimal 
for re-id in a typically unlabelled target domain due to the inevitable presence of 
domain shift in real-world deployment scenarios.

This leads to the necessity of model unsupervised domain adaptation, as detailed below.





















































  §.§.§ Unsupervised Target Domain Adaptation




We want to adapt a learned TJ-AIDL model to fit
the unlabelled target domain data.
To that end, we exploit the attribute consistency principle
by treating the prediction of attribute branch and 
the embedding of IIA component 
as different attribute perspectives from different domains. 
This idea is based on the observation that,
a well fitted TJ-AIDL model is supposed to have small discrepancy 
between the two different attribute perspectives, for example, the one trained on the source domain 
(Fig. <ref>(a)). 
In other words, their consistency degree suggests how well the model fits a given domain.
This also partially shares the spirit of the cyclic consistency mechanism <cit.>.


Specifically, our objective is to adapt the attribute branch since it is used in re-id deployment.

Hence, we can ignore the updating of the identity branch.

We design the following adaptation algorithm:
(1) We deploy the TJ-AIDL model learned on the source domain 
on unlabelled target person images to obtain the attribute prediction 
p_att,t from the attribute branch.
(2) We then utilise the soft label p_att,t as the pseudo groundtruth to
update both the attribute branch and IIA component
for reducing attribute discrepancy between domains (Fig. <ref>(b)). 
Intuitively, this soft attribute label is needed since
we need to prevent the model drifting overly by maintaining the 
most attribute discriminative power obtained from the source domain.
(3) We adapt the model on the target training data until convergence.






























 §.§ Model Optimisation and Deployment


Optimisation
Our TJ-AIDL model can be trained using the standard Stochastic Gradient Descent algorithm
in end-to-end manner. 

We summarise the training process in Alg. <ref>.



Deployment
Given a TJ-AIDL model trained on a labelled source domain and adapted on the unlabelled
target domain,
we obtain a 1,024-D deep feature representation from the attribute branch (Fig. <ref>(b)).

This feature vector is not only attribute semantic but also identity discriminative.

Hence, we deploy this 1,024-D deep feature 
for person re-id deployment by the L_2 distance
in the target domain.





















§ EXPERIMENTS


Datasets and Evaluation Protocol
We choose four widely adopted person re-id benchmarks for
experimental evaluations  (Fig. <ref>).
We adopt the standard supervised re-id data split settings 
and only use the test data for model evaluation whilst the training part is ignored.

(1) The Market-1501 dataset <cit.> contains 32,668
images of 1,501 pedestrians, each of which was captured
by at most six cameras at a university campus. 
All of the images were cropped by a pedestrian detector
and therefore presenting more challenges to re-id models
due to more background clutters and the misalignment problem.

Evaluation Protocol:
We used the standard training/test split (750/751) 
and evaluated on single-query evaluation settings
<cit.>.


(2) The DukeMTMC-ReID dataset <cit.> contains
2∼426 images per person captured by 8 non-overlapping camera views.
This dataset was constructed from the multi-camera tracking dataset 
DukeMTMC by random selection of manually labelled tracklet bounding boxes. 
Evaluation Protocol:
We followed <cit.> by splitting all 1,404 person identities into two halves 702/702 for model training and test, respectively and testing re-id tasks in the single-query setting.







(3) The VIPeR dataset <cit.> has 632 identities each with two images 
captured from two camera views in different scenarios of illumination, postures and viewpoints.
This dataset is also featured with low resolution therefore giving rise to an extremely 
challenging re-id task. 
Evaluation Protocol:
We randomly split the whole population into two halves as training/test sets.
We repeat 10 times of random split and report the average result.



(4) The PRID dataset <cit.> 
consists of person images from two camera views: View
A captures 385 people, whilst View B contains 749 people.
Only 200 people appear in both views.
Evaluation Protocol:
We use the single shot version
in our experiments
as <cit.>.
In each data split, 100 people with one image from each view
are randomly chosen from the 200 present in both camera
views as the training set, while the remaining 100 of View
A are used as the probe set, and the remaining 649 of View
B are used as gallery. Experiments are repeated over 10
random splits.

For performance metric, we use the cumulative matching characteristic (CMC) and mean Average Precision (mAP). 



























Attribute Annotation
In our evaluations, we use either Market-1501 <cit.> or
DukeMTMC-ReID) dataset <cit.> 
as the source domain, since they provide both identity and attribute labels
(Fig. <ref>).

Specifically, there are 27/23 classes of attributes labelled for Market-1501 / DukeMTMC-ReID <cit.>.

To ensure the unsupervised re-id property, we do not test the Market-1501 when
it is used as the source domain. 
This similarly applies to DukeMTMC-ReID.

































Implementation Details 
We realised the TJ-AIDL model in the Tensorflow framework <cit.>.
The IIA encoder is designed as a 3-FC-layers network with
their output dimensions as 512/128/m 
(m is the number of attribute labels).
A network of a mirror structure is used in the IIA decoder.
We fixed both λ_1 and λ_2 to 10 (Eq. (<ref>))
by scale alignment.

We pre-trained the MobileNet on ImageNet for both identity and attribute branches. 
We used the Adam optimiser <cit.> 
with a learning rate of 0.002 and the default momentum terms β_1=0.5, β_2=0.999. 
We set the mini-batch size to 8. 

	We started with training the identity branch by 100,000 iterations 
	on the source identity labels and 
	then the whole model by 20,000 iterations for both 
	transferable joint learning on the labelled source data and 
	unsupervised domain adaptation on the unlabelled target data.
        


























 §.§ Comparisons to the State-Of-The-Arts

We compare 19 models in three categories of existing unsupervised re-id methods:
(1) Hand-crafted feature based methods without transfer learning: 
SDALF<cit.>  and CPS <cit.>, those
features are designed to be view invariant. Dictionary Learning based
methods DLLR <cit.>, graph-learning-based model
GL <cit.>, sparse representation learning methods
ISR  <cit.>, salience-learning-based GTS
<cit.> and SDC<cit.>. 

(2) 
Source identity knowledge transfer learning based methods:
Dic <cit.>,
RKSL <cit.>,
SAE <cit.>,  
AML <cit.>,
UsNCA <cit.>,
CAMEL <cit.>.

(3) Source identity and attribute knowledge based transfer methods:

kLFDA_N <cit.> 
SADA+kLFDA <cit.>
AdaRSVM  <cit.> 
UDML <cit.>.



Table <ref> shows that:
(1) Our method outperforms clearly 
all existing state-of-the-art models, 
improving the Rank-1 by 
0.6% (38.5-37.9), 9.8% (34.8-25.0), 
3.7% (58.2-54.5), 13.9% (44.3-30.4)
over the best alternative method 
on VIPeR/PRID/Market-1501/DukeMCMT-ReID, respectively.

This suggests the overall performance advantages of the proposed
TJ-AIDL in the capability of multi-source (attribute and identity) information extraction
and fusion for cross-domain unsupervised re-id matching.

(2) When compared to the existing methods of 1^st category (non-learning based) 
the performance margins are even much larger, 
e.g. the Rank-1 boost is
8.9% (38.5-29.6),
9.8% (34.8-25.0),
17.9% (58.2-40.3)
on VIPeR/PRID/Market-1501, respectively.

This indicates the importance of learning from labelled source supervision
in cross-domain re-id scenarios, since hand-crafted features are
not sufficiently generalisable across different domains with varying camera view conditions.

(3) When comparing the methods between 2^nd (identity transfer) and 
3^rd (identity and attribute joint transfer) category,
it is interestingly found that the latter is not necessarily
superior over the former.

This means that using more supervision in cross-domain transfer learning
is non-trivial particularly when the label property is heterogeneous such as
identity and attribute.

This also indirectly suggest the model design advantages of our TJ-AIDL
in exploiting the diverse knowledge in different types of label data
for the more challenging cross-domain re-id tasks in 
the unlabelled target scenario typical in real-world deployments.


Finally, it is worth noting that
the performance advantages by our TJ-AIDL are achieved using
much less supervision data of lower diversity from only one source domain
(16,522 images of 702 identities/classes on DukeMCMT-ReID, or
12,936 images of 751 identities on Market-1501)
than strong existing competitors. 
For example,
the methods of 2^nd category utilise 
7 different person re-id datasets with high domain varieties 
(CUHK03<cit.>, CUHK01<cit.>, PRID, VIPeR, 3DPeS<cit.>, i-LIDS<cit.>, Shinpuhkan<cit.>) 
including a total of 44,685 images and 3,791 identities;

The UDML <cit.> 
exploits three different source domains  
including 46,966 images of 3,246 identities for test on VIPeR (target),
and 
47,096 images of 3,493 identities for test on PRID (target).







The SSDAL <cit.> 

benefits from 10 diverse datasets consisting in 
19,000 images of 8,705 person identities and another
20,000 images of 1,221 person tracklets. 













































 §.§ Comparisons to Alternative Fusion Methods






We compare the TJ-AIDL with two multi-source fusion methods:
(a) 
: 
Independently train a
deep CNN model for either attribute or identity label 
in the source domain 
and
use the concatenated feature vectors of the two models
for re-id matching in the target domain.
(b) : 
A seminal multi-task joint learning CNN framework
subjecting the identity and attribute supervision to
a shared feature representation 

in the end-to-end model training.

For re-id deployment on the target domain, we use the multi-supervision shared feature representation.



Table <ref> shows that: 
(1) 
The TJ-AIDL outperforms both alternative fusion methods.
This suggests a clear advantage of our method 
in exploiting and fusing multiple supervision for 
cross-domain re-id in an unsupervised manner.
(2) 
Our method achieves more performance gain over the competitors 
on the transfer from Market-1501 (source) to DukeMTMC-ReID (target)
than the opposite transfer.
This is expected and reasonable because
relative to Market-1501, person images from DukeMTMC-ReID 
have more changes in image resolution and background clutter due to 
wider camera views and more complex scene layout, 
which means the source information itself from Market-1501
is insufficient to generalise the target DukeMTMC-ReID setting
and therefore leading to a higher need for domain adaptation.
Our model strongly and naturally meets this deployment requirement.
For the opposite transfer from DukeMTMC-ReID to Market-1501,
our model gives less performance gain
since there is a lower need for domain adaptation.






































 §.§ Further Analysis and Discussions





Effect of Joint Attribute and Identity Features
We evaluated the effect of joint attribute and identity features by
comparing their individual re-id performances against that of the joint feature.

We obtain their individual model by training a MobileCNN
using either identity or attribute label only.

Table <ref> shows 
feature representation learned by only one supervision is 
significantly inferior that that by our TJ-AIDL. 
For instance, the TJ-AIDL feature 
outperforms ID Only 
by 13.7%(44.3-30.6) in Rank-1 
and 8.4% (23.0-14.6) in mAP on DukeMCMT-ReID (target);
by 6.6% (58.2-51.6) in Rank-1
and 4.9%(26.5-21.6) in mAP on Market-1501 (target).


These validate the complementary effect of jointly learning
attribute and identity information 
and importantly strong capability of our model in maximising 
this latent information in a more transferable context. We also plot three feature distributions of 10 randomly selected test identities of DukeMTMC-ReID (transferred from Market-1501). Figure <ref> shows that: (1) Neither transferring the knowledge of attributes or identities alone can form per-identity compact clusters; (2) By our TJ-AID that transfers attributes and identities jointly, the feature distributions of 10 test identities are much more separated.

















Effect of Target Domain Adaptation
We evaluated the effect of 
the attribute consistency driven 
domain adaptation on unlabelled target training data. 
Table <ref> shows that 
this adaptation clearly improves the re-id performance for the transfer
of DukeMCMT-ReID → Market-1501
(1.1% Rank-1 boost)
and more significantly for 
the case of Market-1501 → DukeMCMT-ReID
(4.7% Rank-1 boost).

This shares a similar observation and underlying reason as in Table <ref>,


validating the benefit of our method 
in varying cross-domain model adaptation in
improving the model compatibility when deployed
to a new target scenario.
















































§ CONCLUSION


We presented a novel Transferable Joint Attribute-Identity Deep Learning (TJ-AIDL)
for more discriminative joint learning of the identity and attribute supervision from an auxiliary domain

in order to particularly address the scalable unsupervised person re-identification problem
in the context of heterogeneous multi-task joint learning and domain
transfer learning.

In contrast to most existing re-id methods
that either ignore the scalability issue in re-id or 
exploit a straightforward yet sub-optimal multi-task joint learning of multi-supervision, 
the proposed model is capable of transferring and integrating multiple
heterogeneous supervision and maximising their latent compatibility 
for optimal person re-id in a progressive and more transferable means.

This is achieved by introducing an Identity Inferred Attribute space 
for interactive attribute and identity discriminative learning in a two-branches CNN architecture.

Moreover, we introduce an attribute consistency maximisation mechanism
to further discriminatively adapt a learned TJ-AIDL model 
to fit any given target re-id deployment without the need for additional data labelling
and hence very scalable to real-world applications.

Extensive evaluations were conducted on four re-id benchmarks to validate the advantages of
the proposed TJ-AIDL model over a wide range of state-of-the-art methods 
on different re-id task scenarios with various challenges.
We also 
compared the TJ-AIDL model with popular multi-supervision fusion methods and
provided detailed component analysis with insights into 
the performance gain of our model design.




§ ACKNOWLEDGEMENTS


This work was partially supported by the China Scholarship Council, 
Vision Semantics Ltd, Royal Society Newton Advanced Fellowship Programme (NA150459),
and InnovateUK Industrial Challenge Project on Developing and Commercialising Intelligent Video Analytics Solutions for Public Safety.



ieee



