arXiv:1708.08062v2 [cs.CV] 18 Oct 2017

Cross-view Asymmetric Metric Learning for
Unsupervised Person Re-identification

Hong-Xing Yu, Ancong Wu, Wei-Shi Zheng

Code is available at the project page:
https://github.com/KovenYu/CAMEL
For reference of this work, please cite:
Hong-Xing Yu, Ancong Wu, Wei-Shi Zheng. “Cross-view Asymmetric
Metric Learning for Unsupervised Person Re-identification.” Proceedings
of the IEEE International Conference on Computer Vision. 2017.
Bib:
@inproceedings{yu2017cross,
title={Cross-view Asymmetric Metric Learning for Unsupervised Person
Re-identification},
author={Yu, Hong-Xing and Wu, Ancong and Zheng, Wei-Shi},
booktitle={Proceedings of the IEEE International Conference on Computer
Vision},
year={2017}
}

Cross-view Asymmetric Metric Learning for
Unsupervised Person Re-identification
Hong-Xing Yu1,5 , Ancong Wu2 , and Wei-Shi Zheng1,3,4∗
School of Data and Computer Science, Sun Yat-sen University, China
2
School of Electronics and Information Technology, Sun Yat-sen University, China
3
Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education, China
4
Collaborative Innovation Center of High Performance Computing, NUDT, China
5
Guangdong Key Laboratory of Big Data Analysis and Processing, Guangzhou, China
1

xKoven@gmail.com, wuancong@mail2.sysu.edu.cn, wszheng@ieee.org

Abstract
While metric learning is important for Person reidentification (RE-ID), a significant problem in visual
surveillance for cross-view pedestrian matching, existing
metric models for RE-ID are mostly based on supervised
learning that requires quantities of labeled samples in all
pairs of camera views for training. However, this limits
their scalabilities to realistic applications, in which a large
amount of data over multiple disjoint camera views is available but not labelled. To overcome the problem, we propose
unsupervised asymmetric metric learning for unsupervised
RE-ID. Our model aims to learn an asymmetric metric, i.e.,
specific projection for each view, based on asymmetric clustering on cross-view person images. Our model finds a
shared space where view-specific bias is alleviated and thus
better matching performance can be achieved. Extensive
experiments have been conducted on a baseline and five
large-scale RE-ID datasets to demonstrate the effectiveness
of the proposed model. Through the comparison, we show
that our model works much more suitable for unsupervised
RE-ID compared to classical unsupervised metric learning
models. We also compare with existing unsupervised REID methods, and our model outperforms them with notable
margins. Specifically, we report the results on large-scale
unlabelled RE-ID dataset, which is important but unfortunately less concerned in literatures.

1. Introduction
Person re-identification (RE-ID) is a challenging problem focusing on pedestrian matching and ranking across
non-overlapping camera views. It remains an open problem
although it has received considerable exploration recently,
in consideration of its potential significance in security ap-

plications, especially in the case of video surveillance. It
has not been solved yet principally because of the dramatic
intra-class variation and the high inter-class similarity. Existing attempts mainly focus on learning to extract robust
and discriminative representations [33, 23, 19], and learning
matching functions or metrics [38, 14, 18, 22, 19, 20, 26]
in a supervised manner. Recently, deep learning has been
adopted to RE-ID community [1, 32, 28, 27] and has gained
promising results.
However, supervised strategies are intrinsically limited
due to the requirement of manually labeled cross-view training data, which is very expensive [31]. In the context of
RE-ID, the limitation is even pronounced because (1) manually labeling may not be reliable with a huge number of images to be checked across multiple camera views, and more
importantly (2) the astronomical cost of time and money
is prohibitive to label the overwhelming amount of data
across disjoint camera views. Therefore, in reality supervised methods would be restricted when applied to a new
scenario with a huge number of unlabeled data.
To directly make full use of the cheap and valuable unlabeled data, some existing efforts on exploring unsupervised
strategies [8, 35, 29, 13, 21, 24, 30, 12] have been reported,
but they are still not very satisfactory. One of the main reasons is that without the help of labeled data, it is rather difficult to model the dramatic variances across camera views,
such as the variances of illumination and occlusion conditions. Such variances lead to view-specific interference/bias
which can be very disturbing in finding what is more distinguishable in matching people across views (see Figure 1).
In particular, existing unsupervised models treat the samples from different views in the same manner, and thus the
effects of view-specific bias could be overlooked.
In order to better address the problems caused by camera view changes in unsupervised RE-ID scenarios, we pro-

Projected by U1

Camera-1

Clustering

Projected by U2
Camera-2

Original Space

Shared Space

Figure 1. Illustration of view-specific interference/bias and our
idea. Images from different cameras suffer from view-specific interference, such as occlusions in Camera-1, dull illumination in
Camera-2, and the change of viewpoints between them. These factors introduce bias in the original feature space, and therefore unsupervised re-identification is extremely challenging. Our model
structures data by clustering and learns view-specific projections
jointly, and thus finds a shared space where view-specific bias is
alleviated and better performance can be achieved. (Best viewed
in color)

pose a novel unsupervised RE-ID model named Clusteringbased Asymmetric1 MEtric Learning (CAMEL). The ideas
behind are on the two following considerations. First, although conditions can vary among camera views, we assume that there should be some shared space where the data
representations are less affected by view-specific bias. By
projecting original data into the shared space, the distance
between any pair of samples xi and xj is computed as:
d(xi , xj ) = kU T xi − U T xj k2 =

p
(xi − xj )T M (xi − xj ),
(1)

where U is the transformation matrix and M = U U T .
However, it can be hard for a universal transformation to
implicitly model the view-specific feature distortion from
different camera views, especially when we lack label information to guide it. This motivates us to explicitly model
the view-specific bias. Inspired by the supervised asymmetric distance model [4], we propose to embed the asymmetric
metric learning to our unsupervised RE-ID modelling, and
thus modify the symmetric form in Eq. (1) to an asymmetric
one:
d(xpi , xqj ) = kU pT xpi − U qT xqj k2 ,

(2)

where p and q are indices of camera views.
An asymmetric metric is more acceptable for unsupervised RE-ID scenarios as it explicitly models the variances
among views by treating each view differently. By such
an explicit means, we are able to better alleviate the disturbances of view-specific bias.
The other consideration is that since we are not clear
about how to separate similar persons in lack of labeled
data, it is reasonable to pay more attention to better separating dissimilar ones. Such consideration motivates us
to structure our data by clustering. Therefore, we develop
1 “Asymmetric”

means specific transformations for each camera view.

asymmetric metric clustering that clusters cross-view person images. By clustering together with asymmetric modelling, the data can be better characterized in the shared
space, contributing to better matching performance (see
Figure 1).
In summary, the proposed CAMEL aims to learn viewspecific projection for each camera view by jointly learning
the asymmetric metric and seeking optimal cluster separations. In this way, the data from different views is projected
into a shared space where view-specific bias is aligned to an
extent, and thus better performance of cross-view matching
can be achieved.
So far in literatures, the unsupervised RE-ID models
have only been evaluated on small datasets which contain
only hundreds or a few thousands of images. However, in
more realistic scenarios we need evaluations of unsupervised methods on much larger datasets, say, consisting of
hundreds of thousands of samples, to validate their scalabilities. In our experiments, we have conducted extensive
comparison on datasets with their scales ranging widely.
In particular, we combined two existing RE-ID datasets
[37, 36] to obtain a larger one which contains over 230,000
samples. Experiments on this dataset (see Sec. 4.4) show
empirically that our model is more scalable to problems of
larger scales, which is more realistic and more meaningful
for unsupervised RE-ID models, while some existing unsupervised RE-ID models are not scalable due to the expensive cost in either storage or computation.

2. Related Work
At present, most existing RE-ID models are in a supervised manner. They are mainly based on learning distance
metrics or subspace [38, 14, 18, 22, 19, 20, 26], learning
view-invariant and discriminative features [33, 23, 19], and
deep learning frameworks [1, 32, 28, 27]. However, all
these models rely on substantial labeled training data, which
is typically required to be pair-wise for each pair of camera
views. Their performance depends highly on the quality
and quantity of labeled training data. In contrast, our model
does not require any labeled data and thus is free from prohibitively high cost of manually labeling and the risk of incorrect labeling.
To directly utilize unlabeled data for RE-ID, several unsupervised RE-ID models [35, 29, 21, 13, 24] have been
proposed. All these models differ from ours in two aspects. On the one hand, these models do not explicitly exploit the information on view-specific bias, i.e., they treat
feature transformation/quantization in every distinct camera
view in the same manner when modelling. In contrast, our
model tries to learn specific transformation for each camera
view, aiming to find a shared space where view-specific interference can be alleviated and thus better performance can
be achieved. On the other hand, as for the means to learn

a metric or a transformation, existing unsupervised methods for RE-ID rarely consider clustering while we introduce
an asymmetric metric clustering to characterize data in the
learned space. While the methods proposed in [4, 2, 3]
could also learn view-specific mappings, they are supervised methods and more importantly cannot be generalized
to handle unsupervised RE-ID.
Apart from our model, there have been some clusteringbased metric learning models [34, 25]. However, to our best
knowledge, there is no such attempt in RE-ID community
before. This is potentially because clustering is more susceptible to view-specific interference and thus data points
from the same view are more inclined to be clustered together, instead of those of a specific person across views.
Fortunately, by formulating asymmetric learning and further limiting the discrepancy between view-specific transforms, this problem can be alleviated in our model. Therefore, our model is essentially different from these models
not only in formulation but also in that our model is able
to better deal with cross-view matching problem by treating
each view asymmetrically. We will discuss the differences
between our model and the existing ones in detail in Sec.
4.3.

acceptable to relax the original idea: we focus on gathering
similar person images together, and hence separating relatively dissimilar ones. Such goal can be modelled by minimizing an objective function like that of k-means clustering
[10]:
U

3.1. Problem Formulation
Under a conventional RE-ID setting, suppose we have
a surveillance camera network that consists of V camera
views, from each of which we have collected Np (p =
1, · · · , V ) images and thus there are N = N1 + · · · + NV
images in total as training samples.
Let X = [x11 , · · · , x1N1 , · · · , xV1 , · · · , xVNV ] ∈ RM ×N
denote the training set, with each column xpi (i =
1, · · · , Np ; p = 1, · · · , V ) corresponding to an M dimensional representation of the i-th image from the pth camera view. Our goal is to learn V mappings i.e.,
U 1 , · · · , U V , where U p ∈ RM ×T (p = 1, · · · , V ), corresponding to each camera view, and thus we can project
the original representation xpi from the original space RM
into a shared space RT in order to alleviate the view-specific
interference.

3.2. Modelling
Now we are looking for some transformations to map
our data into a shared space where we can better separate
the images of one person from those of different persons.
Naturally, this goal can be achieved by narrowing intraclass discrepancy and meanwhile pulling the centers of all
classes away from each other. In an unsupervised scenario,
however, we have no labeled data to tell our model how it
can exactly distinguish one person from another who has a
confusingly similar appearance with him. Therefore, it is

kU T xi − ck k2 ,

(3)

k=1 i∈Ck

where K is the number of clusters, ck denotes the centroid
of the k-th cluster and Ck = {i|U T xi ∈ k-th cluster}.
However, clustering results may be affected extremely
by view-specific bias when applied in cross-view problems.
In the context of RE-ID, the feature distortion could be
view-sensitive due to view-specific interference like different lighting conditions and occlusions [4]. Such interference might be disturbing or even dominating in searching
the similar person images across views during clustering
procedure. To address this cross-view problem, we learn
specific projection for each view rather than a universal one
to explicitly model the effect of view-specific interference
and to alleviate it. Therefore, the idea can be further formulated by minimizing an objective function below:
min

3. Methodology

K X
X

min Fintra =

U 1 ,··· ,U V

s.t.

Fintra =

K X
X

kU pT xpi − ck k2

k=1 i∈Ck

U

pT

p

p

Σ U =I

(4)

(p = 1, · · · , V ),

where the notation is similar to Eq. (3), with p denotes the
view index, Σp = X p X pT /Np + αI and I represents the
identity matrix which avoids singularity of the covariance
matrix. The transformation U p that corresponds to each
instance xpi is determined by the camera view which xpi
comes from. The quasi-orthogonal constraints on U p ensure that the model will not simply give zero matrices. By
combining the asymmetric metric learning, we actually realize an asymmetric metric clustering on RE-ID data across
camera views.
Intuitively, if we minimize this objective function directly, U p will largely depend on the data distribution from
the p-th view. Now that there is specific bias on each view,
any U p and U q could be arbitrarily different. This result is
very natural, but large inconsistencies among the learned
transformations are not what we exactly expect, because
the transformations are with respect to person images from
different views: they are inherently correlated and homogeneous. More critically, largely different projection basis pairs would fail to capture the discriminative nature of
cross-view images, producing an even worse matching result.
Hence, to strike a balance between the ability to capture
discriminative nature and the capability to alleviate viewspecific bias, we embed a cross-view consistency regularization term into our objective function. And then, in consideration of better tractability, we divide the intra-class

0.8

1

0.5

View1, ID1
View2, ID1
View1, ID2
View2, ID2

0.6
0.4

where

1
View1, ID1
View2, ID1
View1, ID2
View2, ID2

0.5

0.2
0

0



H = h1 , ..., hK ,

0

hT
k hl =

-0.2
-0.5

-0.4

View1, ID1
View2, ID1
View1, ID2
View2, ID2

-0.5

-0.6
-1
-1.5

-1

-0.5

0

0.5

-0.8
1.5 -1.5

1

(a) Original

-1

-0.5

0

0.5

1

-1
-0.8

-0.6

(b) Symmetric

-0.4

-0.2

0

0.2

0.4

0.6

Figure 2. Illustration of how symmetric and asymmetric metric
clustering structure data using our method for the unsupervised
RE-ID problem. The samples are from the SYSU dataset [4]. We
performed PCA for visualization. One shape (triangle or circle)
stands for samples from one view, while one color indicates samples of one person. (a) Original distribution (b) distribution in the
common space learned by symmetric metric clustering (c) distribution in the shared space learned by asymmetric metric clustering. (Best viewed in color)

term by its scale N , so that the regulating parameter would
not be sensitive to the number of training samples. Thus,
our optimization task becomes
min

U 1 ,··· ,U V

Fobj =

 1
x1
0

f=  .
X
 ..
0

s.t.

U

p

p

Σ U =I

k=1

k=1 i∈Ck






(9)

xVNV
(10)
(11)

(12)

k

As for the second term, we can also rewrite it as follow:

For convenience, we denote yi = U pT xpi . Then we have
Y ∈ RT ×N , where each column yi corresponds to the projected new representation of that from X. For optimization,
we rewrite our objective function in a more compact form.
The first term can be rewritten as follow [6]:
kyi − ck k2 =



1
fT U
eU
e T X)
f − 1 Tr(H T X
fT U
eU
e T XH).
f
= Tr(X
N
N

(p = 1, · · · , V ),

3.3. Optimization

X

0
0
..
.

K
1 XX
kyi − ck k2
N
i∈C

where λ is the cross-view regularizer and k·kF denotes the
Frobenius norm of a matrix. We call the above model the
Clustering-based Asymmetric MEtric Learning (CAMEL).
To illustrate the differences between symmetric and
asymmetric metric clustering in structuring data in the REID problem, we further show the data distributions in Figure
2. We can observe from Figure 2 that the view-specific bias
is obvious in the original space: triangles in the upper left
and circles in the lower right. In the common space learned
by symmetric metric clustering, the bias is still obvious. In
contrast, in the shared space learned by asymmetric metric
clustering, the bias is alleviated and thus the data is better characterized according to the identities of the persons,
i.e., samples of one person (one color) gather together into
a cluster.

1
N

···
···
..
.
···

and thus Eq. (6) becomes

(5)

K
X

x1N1 0 · · ·
0
0
x21 · · · x2N2
..
..
..
..
.
.
.
.
0
0 ···
0


e = U 1T , · · · , U V T T ,
U
e T X,
f
Y =U

p6=q

k

···
···
..
.
···

so that

K
X p
1 XX
=
kU pT xpi − ck k2 + λ
kU − U q k2F
N
i∈C
pT

(8)

is an indicator vector with the i-th entry corresponding to
the instance yi , indicating that yi is in the k-th cluster if the
corresponding entry does not equal zero. Then we construct

1
Fintra + λFconsistency
N

k=1

(7)


T √
hk = 0, · · · , 0, 1, · · · , 1, 0, · · · , 0, 1, · · · / nk

0.8

(c) Asymmetric

(
0 k 6= l
1 k=l

1
[Tr(Y T Y ) − Tr(H T Y T Y H)],
N
(6)

λ

X p
e T DU
e ),
kU − U q k2F = λTr(U

(13)

p6=q

where

(V − 1)I
 −I

D=
..

.
−I

−I
(V − 1)I
..
.
−I

−I
−I
..
.
−I

···
···
..
.
···


−I
−I 

 . (14)
..

.
(V − 1)I

Then, it is reasonable to relax the constraints
U pT Σp U p = I

(p = 1, · · · , V )

(15)

e TΣ
eU
e = V I,
U pT Σp U p = U

(16)

to
V
X
p=1

e = diag(Σ1 , · · · , ΣV ) because what we expect
where Σ
is to prevent each U p from shrinking to a zero matrix. The
relaxed version of constraints is able to satisfy our need, and
it bypasses trivial computations.
By now we can rewrite our optimization task as follow:
1
fT U
eU
e T X)
f + λTr(U
e T DU
e)
Tr(X
N
1
fT U
eU
e T XH)
f
− Tr(H T X
N
e TΣ
eU
e = V I.
s.t.
U

min Fobj =
e
U

(17)

It is easy to realize from Eq. (5) that our objective function is highly non-linear and non-convex. Fortunately, in
the form of Eq. (17) we can find that once H is fixed, Lagrange’s method can be applied to our optimization task.
And again from Eq. (5), it is exactly the objective of ke is fixed [10]. Thus, we can adopt
means clustering once U
an alternating algorithm to solve the optimization problem.
e . Now we see how we optimize
Fix H and optimize U
e
U . After fixing H and applying the method of Lagrange
multiplier, our optimization task (17) is transformed into an
eigen-decomposition problem as follow:
Gu = γu,

(18)

Algorithm 1: CAMEL

1
2

3

e K,  = 10−8
Input : X,
e
Output: U
e to initialize
Conduct k-means clustering with respect to each column of X
H according to Eq. (7) and (8).
Fix H and solve the eigen-decomposition problem described by Eq. (18) and
e.
(19) to construct U
while decrement of Fobj >  & maximum iteration unreached do
• Construct Y according to Eq. (11).
e and conduct k-means clustering with respect to each column
• Fix U
of Y to update H according to Eq. (7) and (8).
• Fix H and solve the eigen-decomposition problem described by
e.
Eq. (18) and (19) to update U

4

end

where γ is the Lagrange multiplier (and also is the eigenvalue here) and
T fT
fX
fT − 1 XHH
f
e −1 (λD + 1 X
X ).
G=Σ
N
N

(19)

e can be obtained by solving this eigenThen, U
decomposition problem.
e and optimize H. As for the optimization of H,
Fix U
e and conduct k-means clustering in the
we can simply fix U
learned space. Each column of H, hk , is thus constructed
according to the clustering result.
Based on the analysis above, we can now propose the
main algorithm of CAMEL in Algorithm 1. We set maxie , we decompose it
mum iteration to 100. After obtaining U
1
V
back into {U , · · · , U }. The algorithm is guaranteed to
convergence, as given in the following proposition:
Proposition 1. In Algorithm 1, Fobj is guaranteed to convergence.
e is fixed, if H is the loProof. In each iteration, when U
cal minimizer, k-means remains H unchanged, otherwise
e has a
it seeks the local minimizer. When H is fixed, U
closed-form solution which is the global minimizer. Therefore, the Fobj decreases step by step. As Fobj ≥ 0 has a
lower bound 0, it is guaranteed to convergence.

4. Experiments
4.1. Datasets
Since unsupervised models are more meaningful when
the scale of problem is larger, our experiments were conducted on relatively big datasets except VIPeR [9] which
is small but widely used. Various degrees of view-specific
bias can be observed in all these datasets (see Figure 3).
The VIPeR dataset contains 632 identities, with two images captured from two camera views of each identity.
The CUHK01 dataset [16] contains 3,884 images of 971
identities captured from two disjoint views. There are two
images of every identity from each view.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 3. Samples of the datasets. Every two images in a column
are from one identity across two disjoint camera views. (a) VIPeR
(b) CUHK01 (c) CUHK03 (d) SYSU (e) Market (f) ExMarket.
(Best viewed in color)

Dataset

VIPeR

CUHK01 CUHK03

SYSU

Market

ExMarket

# Samples
# Views

1,264
2

3,884
2

24,448
2

32,668
6

236,696
6

13,164
6

Table 1. Overview of dataset scales. “#” means “the number of”.

The CUHK03 dataset [17] contains 13,164 images of
1,360 pedestrians captured from six surveillance camera
views. Besides hand-cropped images, samples detected by
a state-of-the-art pedestrian detector are provided.
The SYSU dataset [4] includes 24,448 RGB images of 502
persons under two surveillance cameras. One camera view
mainly captured the frontal or back views of persons, while
the other observed mostly the side views.
The Market-1501 dataset [37] (Market) contains 32,668
images of 1,501 pedestrians, each of which was captured
by at most six cameras. All of the images were cropped by
a pedestrian detector. There are some bad-detected samples
in this datasets as distractors as well.
The ExMarket dataset2 . In order to evaluate unsupervised
RE-ID methods on even larger scale, which is more realistic, we further combined the MARS dataset [36] with
Market. MARS is a video-based RE-ID dataset which con2 Demo code for the model and the ExMarket dataset can be found on
https://github.com/KovenYu/CAMEL.

tains 20,715 tracklets of 1,261 pedestrians. All the identities
from MARS are of a subset of those from Market. We then
took 20% frames (each one in every five successive frames)
from the tracklets and combined them with Market to obtain
an extended version of Market (ExMarket). The imbalance
between the numbers of samples from the 1,261 persons and
other 240 persons makes this dataset more challenging and
realistic. There are 236,696 images in ExMarket in total,
and 112,351 images of them are of training set. A brief
overview of the dataset scales can be found in Table 1.

4.2. Settings
Experimental protocols: A widely adopted protocol was
followed on VIPeR in our experiments [19], i.e., randomly
dividing the 632 pairs of images into two halves, one of
which was used as training set and the other as testing set.
This procedure was repeated 10 times to offer average performance. Only single-shot experiments were conducted.
The experimental protocol for CUHK01 was the same as
that in [19]. We randomly selected 485 persons as training set and the other 486 ones as testing set. The evaluating procedure was repeated 10 times. Both multi-shot and
single-shot settings were conducted.
The CUHK03 dataset was provided together with its recommended evaluating protocol [17]. We followed the provided protocol, where images of 1,160 persons were chosen
as training set, images of another 100 persons as validation
set and the remainders as testing set. This procedure was repeated 20 times. In our experiments, detected samples were
adopted since they are closer to real-world settings. Both
multi-shot and single-shot experiments were conducted.
As for the SYSU dataset, we randomly picked 251
pedestrians’ images as training set and the others as testing
set. In the testing stage, we basically followed the protocol
as in [4]. That is, we randomly chose one and three images
of each pedestrian as gallery for single-shot and multi-shot
experiments, respectively. We repeated the testing procedure by 10 times.
Market is somewhat different from others. The evaluation protocol was also provided along with the data [37].
Since the images of one person came from at most six
views, single-shot experiments were not suitable. Instead,
multi-shot experiments were conducted and both cumulative matching characteristic (CMC) and mean average precision (MAP) were adopted for evaluation [37]. The protocol of ExMarket was identical to that of Market since the
identities were completely the same as we mentioned above.
Data representation: In our experiments we used the deeplearning-based JSTL feature proposed in [32]. We implemented it using the 56-layer ResNet [11], which produced
64-D features. The original JSTL was adopted to our implementation to extract features on SYSU, Market and ExMarket. Note that the training set of the original JSTL contained

VIPeR, CUHK01 and CUHK03, violating the unsupervised
setting. So we trained a new JSTL model without VIPeR
in its training set to extract features on VIPeR. The similar
procedures were done for CUHK01 and CUHK03.
Parameters: We set λ, the cross-view consistency regularizer, to 0.01. We also evaluated the situation when λ goes
to infinite, i.e., the symmetric version of our model in Sec.
4.4, to show how important the asymmetric modelling is.
Regarding the parameter T which is the feature dimension after the transformation learned by CAMEL, we set T
equal to original feature dimension i.e., 64, for simplicity.
In our experiments, we found that CAMEL can align data
distributions across camera views even without performing
any further dimension reduction. This may be due to the
fact that, unlike conventional subspace learning models, the
transformations learned by CAMEL are view-specific for
different camera views and always non-orthogonal. Hence,
the learned view-specific transformations can already reduce the discrepancy between the data distributions of different camera views.
As for K, we found that our model was not sensitive to
K when N  K and K was not too small (see Sec. 4.4),
so we set K = 500. These parameters were fixed for all
datasets.

4.3. Comparison
Unsupervised models are more significant when applied
on larger datasets. In order to make comprehensive and
fair comparisons, in this section we compare CAMEL with
the most comparable unsupervised models on six datasets
with their scale orders varying from hundreds to hundreds
of thousands. We show the comparative results measured
by the rank-1 accuracies of CMC and MAP (%) in Table 2.
Comparison to Related Unsupervised RE-ID Models. In
this subsection we compare CAMEL with the sparse dictionary learning model (denoted as Dic) [13], sparse representation learning model ISR [21], kernel subspace learning
model RKSL [30] and sparse auto-encoder (SAE) [15, 5].
We tried several sets of parameters for them, and report the
best ones. We also adopt the Euclidean distance which is
adopted in the original JSTL paper [32] as a baseline (denoted as JSTL).
From Table 2 we can observe that CAMEL outperforms
other models on all the datasets on both settings. In addition, we can further see from Figure 4 that CAMEL outperforms other models at any rank. One of the main reasons
is that the view-specific interference is noticeable in these
datasets. For example, we can see in Figure 3(b) that on
CUHK01, the changes of illumination are extremely severe
and even human beings may have difficulties in recognizing
the identities in those images across views. This impedes
other symmetric models from achieving higher accuracies,
because they potentially hold an assumption that the invari-

SS

Dic [13]
ISR [21]
RKSL [30]
SAE [15]
JSTL [32]

29.9
27.5
25.8
20.7
25.7

CUHK01 CUHK03

SYSU

Market

ExMarket

SS/MS

SS/MS

MS

MS

49.3/52.9
53.2/55.7
45.4/50.1
45.3/49.9
46.3/50.6

SS/MS
27.4/36.5
31.1/38.5
25.8/34.8
21.2/30.5
24.7/33.2

21.3/28.6
23.2/33.8
17.6/23.0
18.0/24.2
19.9/25.6

50.2(22.7) 52.2(21.2)
40.3(14.3)
34.0(11.0)
42.4(16.2) 44.0(15.1)
44.7(18.4) 46.4(16.7)

95

70

90
85

60
Matching Accuracy (%)

VIPeR

Setting

Matching Accuracy (%)

Dataset

DIC
ISR
UsNCA
AML
SAE
RKSL
JSTL
CAMEL

50

40

30

46.8/51.1 22.2/31.4 20.9/26.4 44.7(18.4) 46.2(16.2)
47.0/51.7 19.8/29.6 21.1/27.2 45.2(18.9)
-

CAMEL

30.9

57.3/61.9 31.9/39.4 30.8/36.8 54.5(26.3) 55.9(23.9)

Table 2. Comparative results of unsupervised models on the six
datasets, measured by rank-1 accuracies and MAP (%). “-” means
prohibitive time consumption due to time complexities of the
models. “SS” represents single-shot setting and “MS” represents
multi-shot setting. For Market and ExMarket, MAP is also provided in the parentheses due to the requirement in the protocol
[37]. Such a format is also applied in the following tables.

15

70
65
60

50
45

20

5

(a) VIPeR

15

20

80
70

70
60

60

DIC
ISR
UsNCA
AML
SAE
RKSL
JSTL
CAMEL

50

40

30

DIC
ISR
UsNCA
AML
SAE
RKSL
JSTL
CAMEL

50

40

30

20

20
5

10
Rank

15

5

20

(c) CUHK03

10
Rank

15

20

(d) SYSU

85

80

80

75
Matching Accuracy (%)

75
Matching Accuracy (%)

ant and discriminative information can be retained and exploited through a universal transformation for all views. But
CAMEL relaxes this assumption by learning an asymmetric
metric and then can outperform other models significantly.
In Sec. 4.4 we will see the performance of CAMEL would
drop much when it degrades to a symmetric model.
Comparison to Clustering-based Metric Learning Models. In this subsection we compare CAMEL with a typical
model AML [34] and a recently proposed model UsNCA
[25]. We can see from Fig. 4 and Table 2 that compared
to them, CAMEL achieves noticeable improvements on all
the six datasets. One of the major reasons is that they do not
consider the view-specific bias which can be very disturbing in clustering, making them unsuitable for RE-ID problem. In comparison, CAMEL alleviates such disturbances
by asymmetrically modelling. This factor contributes to the
much better performance of CAMEL.
Comparison to the State-of-the-Art. In the last subsections, we compared with existing unsupervised RE-ID
methods using the same features. In this part, we also compare with the results reported in literatures. Note that most
existing unsupervised RE-ID methods have not been evaluated on large datasets like CUHK03, SYSU, or Market,
so Table 3 only reports the comparative results on VIPeR
and CUHK01. We additionally compared existing unsupervised RE-ID models, including the hand-craft-feature-based
SDALF [8] and CPS [7], the transfer-learning-based UDML
[24], graph-learning-based model (denoted as GL) [12], and
local-salience-learning-based GTS [29] and SDC [35]. We
can observe from Table 3 that our model CAMEL can outperform the state-of-the-art by large margins on CUHK01.
Comparison to Supervised Models. Finally, in order to
see how well CAMEL can approximate the performance of
supervised RE-ID, we additionally compare CAMEL with
its supervised version (denoted as CAMELs ) which is easily

10
Rank

(b) CUHK01

Matching Accuracy (%)

23.1
24.3

Matching Accuracy (%)

AML [34]
UsNCA [25]

10
Rank

DIC
ISR
UsNCA
AML
SAE
RKSL
JSTL
CAMEL

75

55

20
5

80

70

DIC
ISR
UsNCA
AML
SAE
RKSL
JSTL
CAMEL

65
60
55
50
45

70
65
60

DIC
AML
SAE
JSTL
CAMEL

55
50
45

40
5

10
Rank

15

5

20

(e) Market

10
Rank

15

20

(f) ExMarket

Figure 4. CMC curves. For CUHK01, CUHK03 and SYSU, we
take the results under single-shot setting as examples. Similar patterns can be observed on multi-shot setting.

Model

SDALF
[8]

CPS
[7]

UDML
[24]

GL
[12]

GTS
[29]

SDC
[35]

CAMEL

VIPeR
CUHK01

19.9
9.9

22.0
-

31.5
27.1

33.5
41.0

25.2
-

25.8
26.6

30.9
57.3

Table 3. Results compared to the state-of-the-art reported in literatures, measured by rank-1 accuracies (%). “-” means no reported
result.

derived by substituting the clustering results by true labels,
and three standard supervised models, including the widely
used KISSME [14], XQDA [19], the asymmetric distance
model CVDCA [4]. The results are shown in Table 4. We
can see that CAMELs outperforms CAMEL by various degrees, indicating that label information can further improve
CAMEL’s performance. Also from Table 4, we notice that
CAMEL can be comparable to other standard supervised
models on some datasets like CUHK01, and even outperform some of them. It is probably because the used JSTL
model had not been fine-tuned on the target datasets: this
was for a fair comparison with unsupervised models which
work on completely unlabelled training data. Nevertheless,
this suggests that the performance of CAMEL may not be

Dataset

SYSU

Market

ExMarket

Dataset

Setting

VIPeR CUHK01 CUHK03
SS

SS/MS

SS/MS

SS/MS

MS

MS

Setting

VIPeR CUHK01 CUHK03
SS

KISSME [14]
XQDA [19]
CVDCA [4]
CAMELs

28.4
28.9
37.6
33.7

53.0/57.1
54.3/58.2
57.1/60.9
58.5/62.7

37.8/45.4
36.7/43.7
37.0/44.6
45.1/53.5

24.7/31.8
25.2/31.7
31.1/38.9
31.6/37.6

51.1(24.5)
50.8(24.4)
52.6(25.3)
55.0(27.1)

48.0(18.3)
47.4(18.1)
51.5(22.6)
56.1(24.1)

CMEL
CAMEL

27.5
30.9

CAMEL

30.9

57.3/61.9 31.9/39.4 30.8/36.8 54.5(26.3) 55.9(23.9)

SS/MS

SS/MS

SYSU

Market

ExMarket

SS/MS

MS

MS

52.5/54.9 29.8/37.5 25.4/30.9 47.6(21.5) 48.7(20.0)
57.3/61.9 31.9/39.4 30.8/36.8 54.5(26.3) 55.9(23.9)

Table 5. Performances of CAMEL compared to its symmetric version, denoted as CMEL.

Table 4. Results compared to supervised models using the same
JSTL features.

far below the standard supervised RE-ID models.

4.4. Further Evaluations
The Role of Asymmetric Modeling. We show what is going to happen if CAMEL degrades to a common symmetric model in Table 5. Apparently, without asymmetrically
modelling each camera view, our model would be worsen
largely, indicating that the asymmetric modeling for clustering is rather important for addressing the cross-view matching problem in RE-ID as well as in our model.
Sensitivity to the Number of Clustering Centroids. We
take CUHK01, Market and ExMarket datasets as examples
of different scales (see Table 1) for this evaluation. Table 6
shows how the performance varies with different numbers
of clustering centroids, K. It is obvious that the performance only fluctuates mildly when N  K and K is not
too small. Therefore CAMEL is not very sensitive to K especially when applied to large-scale problems. To further
explore the reason behind, we show in Table 7 the rate of
clusters which contain more than one persons, in the initial
stage and convergence stage in Algorithm 1. We can see
that (1) in spite of that K is varying, there is always a number of clusters containing more than one persons in both
the initial stage and convergence stage. This indicates that
our model works without the requirement of perfect clustering results. And (2), although the number is various, in
the convergence stage the number is consistently decreased
compared to initialization stage. This shows that the cluster
results are improved consistently. These two observations
suggests that the clustering should be a mean to learn the
asymmetric metric, rather than an ultimate objective.
Adaptation Ability to Different Features. At last, we
show that CAMEL can be effective not only when adopting deep-learning-based JSTL features. We additionally
adopted the hand-crafted LOMO feature proposed in [19].
We performed PCA to produce 512-D LOMO features, and
the results are shown in Table 8. Among all the models, the
results of Dic and ISR are the most comparable (Dic and
ISR take all second places). So for clarity, we only compare
CAMEL with them and L2 distance as baseline. From the
table we can see that CAMEL can outperform them.

K

250

500

750

1000

1250

CUHK01
Market
ExMarket

56.59
54.48
55.49

57.35
54.45
55.87

56.26
54.54
56.17

55.12
54.48
55.93

52.75
54.48
55.67

Table 6. Performances of CAMEL when the number of clusters,
K, varies. Measured by single-shot rank-1 accuracies (%) for
CUHK01 and multi-shot for Market and ExMarket.

K

250

500

750

1000

1250

Initial Stage
Convergence Stage

77.6%
55.8%

57.0%
34.3%

26.3%
18.2%

11.6%
7.2%

6.0%
4.8%

Table 7. Rate of clusters containing similar persons on CUHK01.
Similar trend can be observed on other datasets.

Dataset

VIPeR

Setting

SS

Dic [13]
ISR [21]
L2

15.8
20.8
11.6

19.6/23.6 8.6/13.4 14.2/24.4 32.8(12.2) 33.8(12.2)
22.2/27.1 16.7/20.7 11.7/21.6 29.7(11.0)
14.0/18.6 7.6/11.6 10.8/18.9 27.4(8.3) 27.7(8.0)

26.4

30.0/36.2 17.3/23.4 23.6/35.6 41.4(14.1) 42.2(13.7)

CAMEL

CUHK01 CUHK03

SYSU

Market

ExMarket

SS/MS

SS/MS

MS

MS

SS/MS

Table 8. Results using 512-D LOMO features.

5. Conclusion
In this work, we have shown that metric learning can be
effective for unsupervised RE-ID by proposing clusteringbased asymmetric metric learning called CAMEL. CAMEL
learns view-specific projections to deal with view-specific
interference, and this is based on existing clustering (e.g.,
the k-means model demonstrated in this work) on RE-ID
unlabelled data, resulting in an asymmetric metric clustering. Extensive experiments show that our model can outperform existing ones in general, especially on large-scale
unlabelled RE-ID datasets.

Acknowledgement
This work was supported partially by the National Key Research and Development Program of
China (2016YFB1001002), NSFC(61522115, 61472456,
61573387, 61661130157, U1611461), the Royal Society
Newton Advanced Fellowship (NA150459), Guangdong
Province Science and Technology Innovation Leading Talents (2016TX03X157).

References
[1] E. Ahmed, M. Jones, and T. K. Marks. An improved deep
learning architecture for person re-identification. In CVPR,
2015.
[2] L. An, M. Kafai, S. Yang, and B. Bhanu. Reference-based
person re-identification. In AVSS, 2013.
[3] L. An, M. Kafai, S. Yang, and B. Bhanu. Person reidentification with reference descriptor. TCSVT, 2015.
[4] Y.-C. Chen, W.-S. Zheng, J.-H. Lai, and P. Yuen. An asymmetric distance model for cross-view feature mapping in person re-identification. TCSVT, 2015.
[5] A. Coates, H. Lee, and A. Y. Ng. An analysis of single-layer
networks in unsupervised feature learning. Ann Arbor, 2010.
[6] C. H. Q. Ding and X. He. On the equivalence of nonnegative
matrix factorization and spectral clustering. In ICDM, 2005.
[7] S. C. Dong, M. Cristani, M. Stoppa, L. Bazzani, and
V. Murino. Custom pictorial structures for re-identification.
In BMVC, 2011.
[8] M. Farenzena, L. Bazzani, A. Perina, V. Murino, and
M. Cristani. Person re-identification by symmetry-driven accumulation of local features. In CVPR, 2010.
[9] D. Gray, S. Brennan, and H. Tao. Evaluating appearance
models for recognition, reacquisition, and tracking. In PETS,
2007.
[10] J. A. Hartigan. Clustering algorithms. John Wiley & Sons
Inc, 1975.
[11] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning
for image recognition. In CVPR, 2016.
[12] E. Kodirov, T. Xiang, Z. Fu, and S. Gong. Person reidentification by unsupervised\ ell 1 graph learning. In
ECCV, 2016.
[13] E. Kodirov, T. Xiang, and S. Gong. Dictionary learning with
iterative laplacian regularisation for unsupervised person reidentification. In BMVC, 2015.
[14] M. Köstinger, M. Hirzer, P. Wohlhart, P. M. Roth, and
H. Bischof. Large scale metric learning from equivalence
constraints. In CVPR, 2012.
[15] H. Lee, C. Ekanadham, and A. Y. Ng. Sparse deep belief net
model for visual area v2. In NIPS, 2008.
[16] W. Li, R. Zhao, and X. Wang. Human reidentification with
transferred metric learning. In ACCV, 2012.
[17] W. Li, R. Zhao, T. Xiao, and X. Wang. Deepreid: Deep filter
pairing neural network for person re-identification. In CVPR,
2014.
[18] Z. Li, S. Chang, F. Liang, T. S. Huang, L. Cao, and J. R.
Smith. Learning locally-adaptive decision functions for person verification. In CVPR, 2013.
[19] S. Liao, Y. Hu, X. Zhu, and S. Z. Li. Person re-identification
by local maximal occurrence representation and metric
learning. In CVPR, 2015.
[20] S. Liao and S. Z. Li. Efficient psd constrained asymmetric
metric learning for person re-identification. In ICCV, 2015.
[21] G. Lisanti, I. Masi, A. D. Bagdanov, and A. Del Bimbo. Person re-identification by iterative re-weighted sparse ranking.
TPAMI, 2015.

[22] G. Lisanti, I. Masi, and A. Del Bimbo. Matching people
across camera views using kernel canonical correlation analysis. In ICDSC, 2014.
[23] B. Ma, Y. Su, and F. Jurie. Covariance descriptor based
on bio-inspired features for person re-identification and face
verification. IVC, 2014.
[24] P. Peng, T. Xiang, Y. Wang, M. Pontil, S. Gong, T. Huang,
and Y. Tian. Unsupervised cross-dataset transfer learning for
person re-identification. In CVPR, 2016.
[25] C. Qin, S. Song, G. Huang, and L. Zhu. Unsupervised neighborhood component analysis for clustering. Neurocomputing, 2015.
[26] Y. Shen, W. Lin, J. Yan, M. Xu, J. Wu, and J. Wang. Person
re-identification with correspondence structure learning. In
ICCV, 2015.
[27] R. R. Varior, M. Haloi, and G. Wang. Gated siamese
convolutional neural network architecture for human reidentification. In ECCV, 2016.
[28] F. Wang, W. Zuo, L. Lin, D. Zhang, and L. Zhang. Joint
learning of single-image and cross-image representations for
person re-identification. In CVPR, 2016.
[29] H. Wang, S. Gong, and T. Xiang. Unsupervised learning
of generative topic saliency for person re-identification. In
BMVC, 2014.
[30] H. Wang, X. Zhu, T. Xiang, and S. Gong. Towards unsupervised open-set person re-identification. In ICIP, 2016.
[31] X. Wang, W. S. Zheng, X. Li, and J. Zhang. Cross-scenario
transfer person reidentification. TCSVT, 2015.
[32] T. Xiao, H. Li, W. Ouyang, and X. Wang. Learning deep feature representations with domain guided dropout for person
re-identification. In CVPR, 2016.
[33] Y. Yang, J. Yang, J. Yan, S. Liao, D. Yi, and S. Z. Li. Salient
color names for person re-identification. In ECCV, 2014.
[34] J. Ye, Z. Zhao, and H. Liu. Adaptive distance metric learning
for clustering. In CVPR, 2007.
[35] R. Zhao, W. Ouyang, and X. Wang. Person re-identification
by saliency learning. TPAMI, 2016.
[36] L. Zheng, Z. Bie, Y. Sun, J. Wang, C. Su, S. Wang, and
Q. Tian. Mars: A video benchmark for large-scale person
re-identification. In ECCV, 2016.
[37] L. Zheng, L. Shen, L. Tian, S. Wang, J. Wang, and Q. Tian.
Scalable person re-identification: A benchmark. In ICCV,
2015.
[38] W.-S. Zheng, S. Gong, and T. Xiang. Person re-identification
by probabilistic relative distance comparison. In CVPR,
2011.

