JOURNAL OF IEEE TRANSACTIONS ON XX, VOL. X, NO. X, XX 2018

1

Dual Asymmetric Deep Hashing Learning

arXiv:1801.08360v1 [cs.CV] 25 Jan 2018

Jinxing Li, Bob Zhang, Member, IEEE, Guangming Lu, David Zhang*, Fellow, IEEE

Abstract—Due to the impressive learning power, deep learning
has achieved a remarkable performance in supervised hash
function learning. In this paper, we propose a novel asymmetric
supervised deep hashing method to preserve the semantic structure among different categories and generate the binary codes
simultaneously. Specifically, two asymmetric deep networks are
constructed to reveal the similarity between each pair of images
according to their semantic labels. The deep hash functions
are then learned through two networks by minimizing the gap
between the learned features and discrete codes. Furthermore,
since the binary codes in the Hamming space also should keep
the semantic affinity existing in the original space, another
asymmetric pairwise loss is introduced to capture the similarity between the binary codes and real-value features. This
asymmetric loss not only improves the retrieval performance,
but also contributes to a quick convergence at the training
phase. By taking advantage of the two-stream deep structures
and two types of asymmetric pairwise functions, an alternating
algorithm is designed to optimize the deep features and highquality binary codes efficiently. Experimental results on three
real-world datasets substantiate the effectiveness and superiority
of our approach as compared with state-of-the-art.
Index Terms—deep learning, hashing learning, image retrieval,
similarity

I. I NTRODUCTION

W

ITH the rapid growth of multimedia data in search
engines and social networks, how to store these data
and make a fast search when an novel one such as the image
is given, plays a fundamental role in machine learning. Due
to the low storage cost and fast retrieval speed, hashing techniques have attracted much attention and are widely applied
in nearest neighbor search [1] for information retrieval on
large scale datasets. Hashing learning aims to project the data
from the original space into a Hamming space by generating
compact codes. These codes can not only dramatically reduce
the storage overhead and achieve a constant or sub-linear
time complexity in information search, but also preserve the
semantic affinity existing in the original space.
Many hashing methods have been studied [2] [3] [4] [5] [6]
[7] [8] [9] [10]. Generally, these approaches can be roughly
classified into two categories: data-independent and datadependent hashing methods. Locality Sensitive Hashing (LSH)
[2] and its extension Kernelized LSH (KLSH) [3], as the most
typical data-independent hashing methods, were proposed to
J. Li is with the Department of Computing, Hong Kong Polytechnic
University, Hung Hom, Kowloon (e-mail: csjxli@comp.polyu.edu.hk).
B. Zhang is with the Department of Computer and Information Science, University of Macau, Avenida da Universidade, Taipa, Macau (e-mail:
bobzhang@umac.mo).
G. Lu is with Department of Computer Science, Harbin Institute
of Technology Shenzhen Graduate School, Shenzhen, China (e-mail:
luguangm@hit.edu.cn).
D. Zhang is with the Department of Computing, Hong Kong Polytechnic
University, Hung Hom, Kowloon (e-mail: csdzhang@comp.polyu.edu.hk).

obtain the hashing function by using random projections.
Although the designation of these data-independent methods is
quite simple, they often meet a performance degradation when
the length of the binary codes is relatively low. By contrary,
instead of randomly generating the hashing function like LSH
does, data-dependent methods aims to learn a data-specific
hashing function by using the training data, being capable of
generating shorter binary codes but achieving more remarkable performance. Therefore, various data-dependent hashing
approaches containing both unsupervised and supervised have
been proposed. Unsupervised hashing, e.g. Spectral Hashing
[6], Anchor Graph Hashing (AGH) [7], and Discrete Graph
Hashing (DGH) [8] etc., only try to utilize the data structure
to learn compact binary codes to improve the performance. By
taking the label information into account, supervised hashing
methods attempt to map the original data into a compact
Hamming space to preserve the similarity between each pair
samples. Many representative works including Fast Supervised
Hashing (FastH) [5], Kernel Supervised Hashing (KSH) [9],
and Supervised Discrete Hashing (SDH) [10] etc., demonstrate
that supervised hashing methods often obtain an outstanding
performance compared with unsupervised hashing methods.
Thus, we focus on studying the supervised hashing method in
this paper.
Although some traditional supervised hashing methods
achieve a good performance in some applications, most of
them only linearly map the original data into a Hamming space
by using the hand-crafted features, limited their application
for large-scale datasets which have complex distributions.
Fortunately, due to the powerful capability of data representation, deep learning [11] [12] provides a promising way
to jointly represent the data and learn hash codes. Some
existing deep learning based hashing methods have been
studied, such as Deep Supervised Hashing (DSH) [13] and
Deep Pairwise Supervised Hashing (DPSH) [14], etc. These
approaches demonstrate the effectiveness of the end-to-end
deep learning architecture for hashing learning.
Despite the wide applications of deep neural network on
hashing learning, most of them are symmetric structures in
which the similarity between each pair points are estimated
by the Hamming distance between the outputs of the same
hash function [15]. As described in [15], a crucial problem is
that this symmetric scheme would result in the difficulty of optimizing the discrete constraint. Thus in this paper, we propose
a novel asymmetric hashing method to address aforementioned
problem. Note that a similar work was described by Shen et
al. [15], named deep asymmetric pairwise hashing (DAPH).
However, our study is quite distinctive from DAPH. Shen et
al. tried to approximate the similarity affinity by exploiting
two different hashing functions, which can preserve more
similarity information among the real-value features. However,

JOURNAL OF IEEE TRANSACTIONS ON XX, VOL. X, NO. X, XX 2018

DAPH only exploits a simple Euclidean distance, but ignores
the semantic structure between the learned real-value features
and binary codes [16] [17]. One major deficiency is that it
is difficult to efficiently preserve the similarity in the learned
hash functions and discrete codes. Furthermore, in DAPH, two
different types of discrete hash codes corresponding to two
hash functions are estimated in the training time. However
this strategy would enlarge the gap between two schemes,
resulting in a performance degradation. By contrast, we not
only propose a novel asymmetric structure to learn two different hash functions and one consistent binary code for each
sample at the training phase, but also asymmetrically exploit
real-value and multiple integer values, which permits the better
preservation of similarity between the learned features and
hash codes. Experiments show that this novel asymmetric
structure can get a better performance in image retrieval and
quicker convergence at the training stage.
The main contributions of the proposed method are shown
as follows:
(1) A novel asymmetric deep structure are proposed. Two
streams of deep neural networks are trained to asymmetrically
learn two different hash functions. The similarity between each
pair images are utilized through a pairwise loss according to
their semantic/label information.
(2) The similarity between the learned features and binary
codes are also revealed through an additional asymmetric loss.
Real-value features and binary codes are bridged through an
inner product, which alleviates the binary limitation, better
preserves the similarity, and speeds up convergence at the
training phase.
(3) By taking advantage of these two asymmetric properties,
an alternative algorithm is designed to efficiently optimize the
real values and discrete values.
(4) Experimental results on three large-scale datasets substantiate the effectiveness and superiority of our approach as
compared with some existing state-of-the-art hashing methods
in image retrieval.
The rest of this paper is organized as follows. In Section
2, the related works including data-independent and datadependent hashing methods are briefly reviewed. In Section
3, the proposed Dual Asymmetric Deep Hashing Learning
(DADH) is then analyzed, followed by its optimization. In
Section 4, experiments are conducted on three real-world
datasets, and some comparisons, parameter sensitivity analysis
and convergence analysis are discussed. This paper is finally
concluded in Section 5.
II. R ELATED W ORKS
As mentioned before, the hashing method can be roughly
separated into data-independent and data-dependent hashing.
Locality Sensitive Hashing (LSH) [2] aims to use several
hash functions to randomly project the data into a Hamming
space, so as to ensure the probability of collision is much
higher for data points which are close to each other than for
those which are far apart. Consider the non-linearity existing in
many real-world datasets, LSH was generated to accommodate
arbitrary kernel functions (KLSH) in [3]. Some other priors,

2

such as p-stable distributions [18] and shift-invariant kernels
[19], are also embedded to extend LSH for performance
improvement.
Different from data-independent methods, the datadependent methods try to learn more compact codes from
a given dataset to achieve a satisfactory search accuracy.
According to whether the label information is available, datadependent hashing can also be classified into unsupervised and
supervised. Typical learning criteria for unsupervised hashing
methods contains graph learning [6] [7] [8] [20] and error
minimization [4] [21] [22]. Graph Learning: Yair et al. [6]
proved that finding a best code is associated with the problem
of graph partitioning. Thus, a spectral hashing (SH) was proposed to learn the hash function. Another graph hashing named
Anchor Graph Hashing (AGH) was presented by Liu et al.
[7], which is capable of capturing the neighborhood structure
inherent automatically. In order to avoid the high complexity
of existing graph hashing methods, Jiang et al. [20] proposed
a scalable graph hashing (SGH) which can be effectively
applied to the large-scale dataset search. Although SH, AGH
and SGH achieve a satisfactory performance in some datasets,
both of them relax the optimization by discarding the discrete
constraints, which results in an accumulated quantization error.
To address this problem, a discrete graph hashing (DGH)
[8] was proposed, which can find the neighborhood structure
inherent in a discrete code space. Error Minimization: A
typical method is the iterative quantization (ITQ) [4] which
aims to project the data to the vertices of a binary hypercube
and minimize the quantization error. Additionally, the method
in [21] makes a quantization by decomposing the input space
into a Cartesian product of low-dimensional subspaces, dramatically reducing the quantization noise. Different from most
single-bit quantization, a double-bit quantization hashing [23]
was also studied by quantizing each dimension into double
bits.
In contrast to unsupervised hashing methods, supervised
hashing learning utilizes the label information to encourage the
binary codes in the Hamming space to preserve the semantic
relationship existing in the raw data. For instance, Mohammad
et al. [24] introduced a hinge-like loss function to exploit the
semantic information. Besides, Li et al. [25] projected the
raw data into a latent subspace, and the label information is
embedded on this subspace to preserve the semantic structure.
The Jensen Shannon Divergence is also utilized in [26] to
learn the binary codes within a probabilistic framework, in
which an upper bound is derived for various hash functions.
Being similar to KLSH, Liu et al. [9] proposed a supervised
hashing with kernels, in which the similar pairs are minimized
while the dissimilar pairs are maximized. Consider the discrete
constraint, the supervised discrete hashing (SDH) [10] was
proposed to not only preserve the semantic structure, but
also discretely learn the hash codes without any relaxation.
However, this discrete optimization is time-consuming and
unscalable. To tackle this problem, a novel method named
column sample based discrete supervised hashing (COSDISH)
was presented to directly obtain the binary codes from semantic information.
Although various works mentioned above have been stud-

JOURNAL OF IEEE TRANSACTIONS ON XX, VOL. X, NO. X, XX 2018

ied, they only project the data into the Hamming space by
using the hand-crafted features. The main limitation is that
they would meet a performance degradation if the distribution
of a real-world dataset is complex. Fortunately, deep learning
provides a reasonable and promising solution. Liong et al.
[27] used the deep structure to hierarchically and non-linearly
learn the hash codes. Convolutional neural network (CNN)
was first applied by Xia and Yan et al. to the hashing learning
(CNNH) [28], which simultaneously represents the image and
learns a hash function. A novel deep structure [29] was then
proposed by modifying the fully-connected layer in CNNH
to a divide-and-encode module, in which the hash codes can
be obtained bit by bit. Also, Can et al. [30] combined the
quantization model with deep structure to gain a satisfactory
performance in image retrieval. Different from the triple loss
used in some deep hashing methods, Li et al. [14] studied
a pairwise loss (DPSH) which can effectively preserve the
semantic information between each pair outputs. Due to the
power of asymmetric structure, the asymmetric deep hashing
was also studied in recent years. For instance, Shen et al.
[15] (DAPH) tried to learn hash functions in an asymmetric network. However, DAPH only exploits two streams to
preserve the pairwise label information between the deep
neural network outputs, but ignores the similarity between
the real-value features and binary codes. Thus, in this paper,
we propose a novel deep hashing method to not only exploit
the label information between each two outputs through an
asymmetric deep structure, but also semantically associated
the learned real-value features with the binary codes.
III. T HE P ROPOSED M ETHOD
In this section, we first give some notations used in this
paper, as well as the problem definition. The proposed Dual
Asymmetric Deep Hashing Learning (DADH) is then described, followed by its optimization.
A. Notation and Problem Definition
In this paper, since there are two streams in
the proposed method, we use the uppercase letters
X
=
{x1 , · · · , xi , . . . , xN }
∈
RN ×d1 ×d2 ×3 and
N ×d1 ×d2 ×3
Y = {y1 , · · · , yi , . . . , yN } ∈ R
to denote
the input images in the first and second deep neural networks,
respectively, where N is the number of training samples,
d1 and d2 are the length and width for each image. Note
that, although X and Y are represented with different
symbols, both of them denote the same training data. In our
experiments, we only alternatively use training samples X
and Y in the first and second networks. Since our method
is supervised learning, the label information can be used.
Let the uppercase letter S ∈ {−1, +1} denote the similarity
between X and Y and Si,j is the element in the i-th row and
j-th column in S. Let Si,j = 1 if xi and yj share the same
semantic information or label, otherwise Si,j = −1.
Denote the binary codes as B = [b1 , · · · , bi , . . . , bN ]T ∈
N ×k
R
and the k-bit binary code of the i-th sample as
bi ∈ {−1, +1}k×1 . The purpose of our model is to learn
two mapping functions F and G to project X and Y into the

3

TABLE I
T HE NETWORK STRUCTURE OF CNN-F.

Layer
conv1
conv2
conv3
conv4
conv5
full6
full7
full8

Structure
f. 64 × 11 × 11; st. 4×4; pad. 0; LRN.; ×2 pool
f. 265 × 5 × 5; st. 1×1; pad. 2; LRN.; ×2 pool
f. 265 × 3 × 3; st. 1×1; pad. 1
f. 265 × 3 × 3; st. 1×1; pad. 1
f. 265 × 3 × 3; st. 1×1; pad. 1; ×2 pool
4096
4096
→ k-bit hash code

Hamming space B. bi = sign(F(xi )) and bj = sign(G(yj )),
where sign(·) is an element-wise sign function, and sign(x) =
1 if x ≥ 0, otherwise sign(x) = -1. The Hamming distance
distH (bi , bj ) between bi and bj should be as small as
possible if sij = 1 and vice versa. Due to the power of
deep neural network in data representation, we apply the
convolution neural work to learn the hash functions. Specifically, the CNN-F structure [31] is adopted to perform feature
learning. In CNN-F model, there are eight layers including five
convolutional layers as well as three fully-connected layers.
The network structure is listed in Table I, where ”f.” means
the filter, ”st.” means the convolution stride, ”LRN” means the
Local Response Normalization [11]. In order to get the final
binary code, we replace the last layer in CNN-F with a k-D
vector and the k-bit binary codes are obtained through a sign
operation on the output of the last layer. In this paper, CNN-F
model is applied to both streams in our proposed asymmetric
structure.
B. Dual Asymmetric Deep Hashing Learning
The main framework of the proposed method is shown
in Fig.1. As we can see, there are two end-to-end neural
networks to discriminatively represent the inputs. For a pair
of outputs F and G in these two streams, their semantic
information is exploit through a pairwise loss according to
their predefined similarity matrix. Since the purpose is to
obtain hash functions through the deep networks, the binary
code B is also generated by minimizing its distance between
F and G. Furthermore, in order to preserve the similarity
between the learned binary codes and real-value features, and
alleviate the binary limitation, another asymmetric pairwise
loss is introduced by using the inner product of the hash codes
B and learned features F (G).
Denote f (xi , Wf ) ∈ Rk×1 as the output of the i-th sample
in the last layer of the first stream, where Wf is the parameter
of the network. To simplify the notation, we use fi to replace
f (xi , Wf ). Similarly, we can obtain the output gj corresponding to the j-th sample under the parameter Wg in the second
network. Thus, the features F = [f1 , · · · , fi , fn ]T ∈ Rn×k and
G = [g1 , · · · , gi , gn ]T ∈ Rn×k corresponding to the first and
second networks are then gained.
To learn an accurate binary code, we set sign(fi ) and
sign(gi ) to be close to their corresponding hash code bi . A
general way is to minimize the L2 loss between them.
min ksign(fi ) − bi k22 + ksign(gi ) − bi k22 ,
s.t. bi ∈ {−1, +1}

(1)

JOURNAL OF IEEE TRANSACTIONS ON XX, VOL. X, NO. X, XX 2018

4

. ..

. ..

. ..

. ..

U
1, 0,
 0,1,



1, 0,

Real-valued
Data Similarity
Preservation

, 0,1 
, 0, 0 
S

,1,1 

V

Hash Code

Feature Extraction with
different Weights

B
 1
 1
 
 1
 
 
 1
 

Real-valued
Data and
Binary Codes
Similarity
Preservation

. ..

. ..

. ..

. ..

Fig. 1. The framework of the proposed method. Two streams with five convolution layers and two full-connected layers are used for feature extraction. For
the real-valued outputs from these two neural networks, their similarity is preserved by using a pairwise loss. Based on the outputs, a consistent hash code is
generated. Furthermore, an asymmetric loss is introduced to exploit the semantic information between the binary code and real-valued data.

However, it is difficult to make a back-propagation for the
gradient with respect to fi or gi in Eq.(1) since their gradients
are zero anywhere. In this paper, we apply tanh(·) to softly
approximate the sign(·) function. Thus, Eq.(1) is transformed
into
2
2
min ktanh(fi ) − bi k2 + ktanh(gi ) − bi k2 ,
(2)
s.t. bi ∈ {−1, +1}

prove that this kind of asymmetric inner product can quickly
make the network converge to a stable value for the real-valued
features and hash codes.
Jointly taking Eq.(2), Eq.(4) and Eq.(5) into account, the
objective function can be obtained as follows:
min L = tanh(F)BT − kS

F,G,B

Furthermore, to exploit the label information and keep
a consistent similarity between two outputs F and G, the
negative log likelihood of the dual-stream similarities with the
likelihood function is exploited.

σ(Θij )
Sij = 1
p(Sij |tanh(fi ), tanh(gj )) =
(3)
1 − σ(Θij ) Sij = 0
where Θij = 12 tanh(fiT )tanh(gj ), and σ(Θij ) = 1+e1−Θij .
Therefore, the pairwise loss for these two different outputs
is shown as follows.
n
X
min −
(Sij Θij − log(1 + eΘij )
(4)
i,j=1

Although Eq.(2) achieves to approximate discrete codes
and Eq.(4) exploits the intra- and inter-class information, the
similarity between the binary codes and real-value features is
ignored. To tackle this problem, another asymmetric pairwise
loss is introduced.
min tanh(fiT )bj − kSij
s.t. bi ∈ {−1, +1}

2
2

+ tanh(giT )bj − kSij

2
2

,

(5)

In Eq.(5), the similarity between the real-valued data and
binary codes is measured by their inner product. It is easy to
observe that Eq.(5) not only encourages the tanh(fi ) (tanh(gi ))
and bi to be consistent, but also preserve the similarity
between them. Additionally, our experiments in Section 4 also

−τ

n
X

2
F

+ tanh(G)BT − kS

2
F

(Sij Θij − log(1 + eΘij )+

i,j=1
2

2

+ γ(ktanh(F) − BkF + ktanh(G) − BkF )
+ η( tanh(F)T 1

2
F

+ tanh(G)T 1

2
)
F

s.t. bi ∈ {−1, +1}
(6)
where τ , γ and η are the non-negative parameters to make a
trade-off among various terms. Note that the purpose of the
2
2
forth term tanh(F)T 1 F + tanh(G)T 1 F in the objective
function Eq.(6) is to maximize the information provided by
each bit [32]. In detail, this term makes a balance for each bit,
which encourages the number of -1 and +1 to be approximately
similar among all training samples.

C. Optimization
From the objective function Eq.(6), we can see that the
real-valued features as well as the weights in two neural
networks (F, Wf ) / (G, Wg ), and discrete codes B need
to be optimized. Note that this NP-hard problem is highly
non-convex, and it is very difficult to directly get the optimal
solutions. In this paper, we design an efficient algorithm to
optimize them alternatively. Specially, we update one variable
by fixing other variables.

JOURNAL OF IEEE TRANSACTIONS ON XX, VOL. X, NO. X, XX 2018

5

1) Update (F, Wf ) with (G, Wg ) and B fixed: By fixing
(G, Wg ) and B, the objective function Eq.(6) can be transformed to
n
X
2
min tanh(F)BT − kS F − τ
(Sij Θij − log(1 + eΘij )+
F

i,j=1

+ γ(ktanh(F) −

2
BkF

+ η tanh(F)T 1

2
F

(7)
Then the back-propagation is exploited to update (F, Wf ).
Here denote U = tanh(F) and V = tanh(G). The gradient of
the objective function with respect to fi is
n
X
∂L
τ
={
[2bj (bTj ui − kSij ) + (σ(Θij )vj − Sij vj )]
∂fi
2
j=1

+ 2γ(ui − bi ) + 2ηUT 1}

(1 − u2i )

(8)
where
denotes the dot product. After getting the gradient
∂L
∂L
∂fi , the chain rule is used to obtain ∂Wf , and Wf is updated
by using back-propagation.
2) Update (G, Wg ) with (F, Wf ) and B fixed: Similarly,
by fixing (F, Wf ) and B, the back-propagation is exploited to
update (G, Wg ). The gradient of the objective function with
respect to gi is
n
X

∂L
τ
[2bj (bTj vi − kSij ) + (σ(Θij )uj − Sij uj )]
={
∂gi
2
j=1
+ 2γ(vi − bi ) + 2ηVT 1}

(1 − vi2 )

(9)
∂L
,
the
chain
rule
is
used
to
obtain
After getting the gradient ∂g
i
∂L
∂Wg , and Wg is updated by using back-propagation.
3) Update B with (F, Wf ) and (G, Wg ) fixed: By fixing
(F, Wf ) and (G, Wg ), we can get the following formulation.
min L(B) = UBT − kS
B

+ γ(kU −

2
BkF

2
F

+ VBT − kS

+ kV −

2
BkF )

s.t. bi ∈ {−1, +1}
(10)

minL(B) = −2Tr[B(k(UT S + VT S) + γ(UT + VT ))]
B

2
F

+ BVT

2
F

+ const s.t. bi ∈ {−1, +1}
(11)
where ’const’ means a constant value without any association
with B. For the sake of simplicity, let Q = −2k(ST U +
ST V) − 2γ(U + V). Eq.(11) can be simplified to
min L(B) = BUT
B

2
F

+ BVT

2
F

+ Tr[BQT ] + const

s.t. bi ∈ {−1, +1}
(12)
According to Eq.(12) and [17], B can be updated bit by bit.
In other words, we update one column in B with remaining
columns fixed. Let B∗c be the c-th column and B̂c be the
remaining columns in B. So do U∗c , Ûc , V∗c , V̂c , Q∗c , and
Q̂c . Eq.(12) can then be rewrote as:
T
min Tr(B∗c [2(UT∗c Ûc + V∗c
V̂c )B̂Tc + QT∗c ] + const
B∗c

s.t. B ∈ {−1, +1}n×k

Obviously, the optimal solution for B∗c is
B∗c = −sign(2B̂c (ÛTc U∗c + V̂cT V∗c ) + Q∗c )

(14)

After computing B∗c , we update B by replace the c-th column
with B∗c . Then we repeat Eq.(14) until all columns are
updated.
Overall, the optimization of the proposed method is listed
in Algorithm 1.
D. Query

2
F

Then Eq.(10) can be rewrote as:

+ BUT

Algorithm 1 Dual Asymmetric Deep Hashing Learning
(DADH)
Input: Training data X/Y; similarity matrix S; hash code
length k; predefined parameters τ , γ and η.
Output: Hashing functions F and G.
Initialization: Initialize weights of the first seven layers by
using the pretrained ImageNet model; the last layer is
initialized randomly; B is set to be a matrix whose
elements are zero.
1: while not converged or not reach the maximum iteration
do
2:
Update (F, Wf ):
Fix (G, Wg ) and B and update (F, Wf ) using backpropagation according to Eq.(8).
3:
Update (G, Wg ):
Fix (F, Wf ) and B and update (G, Wg ) using backpropagation according to Eq.(9).
4:
Update B:
Fix (F, Wf ) and (G, Wg ) and update B according to
Eq.(14).
5: end while

(13)

When Wf and Wg are learned, the hash functions corresponding to the two neural networks are subsequently obtained. For a given testing image x∗ , two kinds of binary
codes can be computed, which are b∗f = sign(f (x∗ , Wf )) and
b∗g = sign(f (x∗ , Wg )), respectively. Note that since tanh will
not influence the sign of each element at the testing phase,
we do not apply tanh for the output. From the experiments
we find that the performances computed through the first and
second networks are quite similar. To obtain a more robust
result, we use the average of two outputs as the final result in
our experiment.
b∗ = sign(0.5[f (x∗ , Wf ) + g(x∗ , Wg )])

(15)

IV. E XPERIMENTS
In this section, experiments are conducted on three largescale datasets to demonstrate the effectiveness of the proposed
method compared with some state-of-the-art approaches. We
first describe the datasets used in our experiments, followed
by the description of baselines, evaluation protocol, and implementation. We then make a comparison with other methods.
The parameter sensitivity as well as the convergence are
subsequently discussed.

JOURNAL OF IEEE TRANSACTIONS ON XX, VOL. X, NO. X, XX 2018

(a)

(d)

6

(b)

(c)

(e)

(f)

Fig. 2. The Precision-Recall curves computed by LSH, ITQ, DPLM, SDH, SGH, DPSH, ADSH, DAPH, and DADH on the IAPR TC-12 dataset. Figures
from (a) to (f) are associated with the code length 8-bit, 12-bit, 16-bit, 24-bit, 36-bit and 48-bit.

TABLE II
T HE MAP SCORES OBTAINED BY DIFFERENT METHODS ON THE IAPR
TC-12 DATASET.

Method
LSH
ITQ
DPLM
SDH
SGH
DPSH
ADSH
DAPH
DADH

8-bit
32.89
35.83
36.38
36.92
34.48
45.19
44.69
44.33
46.54

12-bit
33.39
36.00
36.86
37.81
35.01
46.03
46.98
44.48
49.27

16-bit
33.25
36.21
37.35
37.45
35.32
46.82
48.25
44.73
50.83

24-bit
34.33
36.55
37.71
38.12
35.49
47.37
49.06
45.12
52.71

36-bit
34.99
36.66
38.64
38.53
35.84
47.97
50.24
45.24
54.47

48-bit
35.15
36.75
38.83
38.27
36.21
48.60
50.59
45.52
55.39

labels, we also define two images be a ground-truth neighbor if
they share at least one same label. Additionally, 2000 images
are randomly selected as the testing data, and the rest is defined
as the retrieval data. Meanwhile, 5000 samples selected from
the retrieval data are used for training.
CIFAR-10 dataset [35] contains 60000 32×32 color images
with ten categories. Each image belongs to one of these ten
classes. Two images will be regarded as semantic neighbor
if they have the same label. Being similar to the setting in
[17], we randomly select 1000 samples to be the testing data.
In order to reduce the training time, we also randomly select
5000 images from the remaining 59000 images as the training
data. The rest is then regarded as the retrieval set.

A. Datasets

B. Baseline and Evaluation Protocol

Three datasets including IAPR TC-12 [33], MIRFLICKR25K [34] and CIFAR-10 [35] are used in this paper.
IAPR TC-12 [33] dataset consists of 20000 images associated with 255 categories. Since some samples have multiple
labels, we set Sij = 1 only if there is at least one same label
for the i-th and j-th sample. In this dataset, 2000 images are
used for testing and 5000 samples selected from the remaining
18000 (retrieval set) points are used for training to greatly
reduce the training time.
MIRFLICKR-25K dataset [34] is composed of 25000 images collected from the Flickr website. According to [32],
20015 images associated with 24 categories are selected. Being
similar to IAPR TC-12 dataset, some images has multiple

To demonstrate the superiority of DADH, some existing
hashing methods are used for comparison, including one
data-independent methods (LSH [2]), four traditional datadependent methods (ITQ [4], DPLM [36], SDH [10], SGH
[20]) and three deep learning based hashing methods (DPSH
[14], ADSH [17], DAPH [15]). Since LSH, ITQ, DPLM,
SDH, and SGH are not deep learning methods, features
should be extracted previously. For these three datasets, we
have extracted the 4096-D CNN feature 512-D GIST feature,
respectively. We have found that these five approaches often
achieve a better performance on the GIST feature. Thus we use
the GIST feature as the input for LSH, ITQ, DPLM, SDH, and
SGH. For DPSH, ADSH, and DAPH, the raw image is used

JOURNAL OF IEEE TRANSACTIONS ON XX, VOL. X, NO. X, XX 2018

7

TABLE III
T HE T OP -500 MAP AND T OP -500 P RECISION SCORES OBTAINED BY DIFFERENT METHODS ON THE IAPR TC-12 DATASET.

Evaluation
Method
LSH
ITQ
DPLM
SDH
SGH
DPSH
ADSH
DAPH
DADH

8-bit
36.19
41.16
42.21
44.08
40.04
57.07
53.70
56.26
57.12

12-bit
37.47
41.91
43.12
45.31
41.54
58.13
58.12
57.98
62.67

MAP@Top500
16-bit 24-bit
37.53
39.83
42.76
43.67
43.89
44.50
46.00
47.30
42.09
42.48
59.94
61.61
61.35
63.25
59.48
61.27
65.15
67.80

36-bit
40.50
44.07
45.86
48.17
43.07
63.05
64.90
62.57
70.11

as the input and all images are resized into 224×224×3. For
all deep learning methods, the CNN-F is used as the network
for feature extraction and the parameters in DPSH and ADSH
are set according to their descriptions in their publications.
Note that, since the code for DAPH is not released, we
implement it with the deep learning toolbox MatConvNet [37]
very carefully. Additionally, the original network structure in
DAPH is not CNN-F which means the parameters in [15] may
be not optimal. Thus, we try our best to tune the parameters
in DAPH.
To quantatively measure the proposed method and other
comparison methods, two widely used metrics containing
mean average precision (MAP) and precision-recall (PR) are
adopted. The definitions of MAP criteria is demonstrated as
follows: Given a query, the average precision (AP) is first
computed by searching a set of R retrieved results.
AP =

1
T

R
X

P (r)δ(r)

(16)

r=1

where T is the total number of document set in retrieved set,
P (r) is the precision of top r retrieved cases, and δ(r) denotes
whether the retrieved sample is relevant (if the instance is a
true neighbor of the query, δ(r) = 1, otherwise δ(r) = 0).
Additionally, being similar to some existing methods [15] [32],
Top-500 MAP and Top-500 Precision are also exploited to
evaluate the superiority of the proposed method.

C. Implementation
We implement DADH with the deep learning toolbox MatConvNet [37] on Titan X GPU. The pre-trained ImageNet
model is used to initialize the first seven layers in each stream
and the weights in the last layer are initialized randomly.
During the training time, we set the mini-batch size to be
64 and divide the learning rate among [10−6 , 10−4 ] into 150
iterations. In other words, the learning rate gradually reduces
from 10−4 to 10−6 and the stochastic gradient descent is used
to update the weights. Based on the cross-validation (a small
set for validation is randomly selected from the training data),
we set γ = 100, η = 10, and τ = 10 in the three datasets. We
will further demonstrate the insensitivity of these parameters
in the following subsection.

48-bit
41.52
44.30
46.42
48.06
43.97
64.49
65.59
63.94
70.93

8-bit
35.04
39.73
40.84
42.45
38.67
55.03
52.31
54.32
55.31

12-bit
36.10
40.30
41.65
43.54
39.70
55.90
56.47
55.44
60.20

Precision@Top500
16-bit 24-bit
36.12
38.10
40.88
41.60
42.33
42.82
43.94
45.00
40.08
40.33
57.73
58.96
58.97
60.29
56.52
57.92
62.52
64.93

36-bit
38.82
41.86
44.02
45.79
40.79
60.15
61.94
58.95
67.13

48-bit
39.55
42.04
44.44
45.49
41.45
61.40
62.50
60.03
67.81

TABLE IV
T HE MAP SCORES OBTAINED BY DIFFERENT METHODS ON THE
MIRFLICKR-25K DATASET.

Method
LSH
ITQ
DPLM
SDH
SGH
DPSH
ADSH
DAPH
DADHL

8-bit
56.06
57.59
60.42
60.17
57.35
73.48
75.39
72.79
77.15

12-bit
56.10
57.57
60.51
60.27
57.54
74.68
76.41
74.70
78.16

16-bit
56.72
57.70
60.53
60.46
57.57
75.58
76.98
74.30
78.64

24-bit
56.82
57.79
60.70
60.67
57.67
76.01
76.59
74.14
79.44

36-bit
57.56
57.84
60.91
60.96
57.80
76.09
76.20
73.81
79.72

48-bit
57.35
57.87
60.79
61.59
57.86
76.05
74.53
73.41
79.26

D. Comparison with Other Methods
1) IAPR TC-12: The MAP scores obtained by different
methods on the IAPR TC-12 dataset are shown in Tab.II.
It is easy to observe that DADH achieves a remarkable
improvement in MAP scores compared with other approaches.
In contrast to the data-independent method LSH, DADH
achieves more than 10%-20% percents higher in MAP scores.
Compared with ITQ, DPLM, SDH and SGH, there is also
an obvious enhancement. Specifically, our proposed method
obtains at least 46.54% MAP score and reaches as high as
55.39% when the bit length is 48, while the best result obtained
by ITQ, DPLM, SDH and SGH is only 38.83%, being far
below than our’s. Referring to DPSH, ADSH and DAPH,
DADH also has more or less improvement in MAP scores.
Particularly, compared with DAPH, the presented approach
gains about or more than 5% enhancement when the bit length
ranges from 12 to 48. In comparison to DPSH and ADSH,
the performance obtained by DADH also has about 3%-5%
improvement when the length of hashing bit is 24, 36, and
48, respectively.
The Top-500 MAP and Top-500 Precision scores on the
IAPR TC-12 dataset are listed in Tab.III. From this table we
can see that the experimental results obtained by the deep
learning based methods including DPSH, ADSH, DAPH and
DADH are remarkably better than that computed by other
traditional approaches. Specifically, there is about 10%-20%
improvement in MAP@Top500 and Precision@Top500 scores
in most cases. In contrast to DPSH, ADSH and DAPH,
the performance achieved by the proposed method DADH

JOURNAL OF IEEE TRANSACTIONS ON XX, VOL. X, NO. X, XX 2018

8

TABLE V
T HE T OP -500 MAP AND T OP -500 P RECISION SCORES OBTAINED BY DIFFERENT METHODS ON THE MIRFLICKR-25K DATASET.

Evaluation
Method
LSH
ITQ
DPLM
SDH
SGH
DPSH
ADSH
DAPH
DADHL

8-bit
57.57
61.00
63.08
65.16
60.74
82.88
82.14
81.08
85.80

12-bit
58.09
61.17
63.84
64.75
61.23
83.84
83.80
84.20
86.83

MAP@Top500
16-bit 24-bit
59.27
59.55
61.32
61.61
64.05
64.36
65.40
65.34
61.48
61.42
84.34
84.84
84.94
84.90
83.71
84.45
86.90
87.42

36-bit
60.84
61.64
64.73
65.57
61.91
85.77
84.20
84.02
87.98

48-bit
60.67
61.85
65.03
66.46
62.04
85.64
82.06
84.07
87.58

8-bit
56.94
59.98
62.36
64.05
59.81
81.85
81.50
80.37
84.73

12-bit
57.25
60.11
63.14
63.93
60.21
83.01
83.18
83.24
85.78

Precision@Top500
16-bit 24-bit
58.50
58.69
60.29
60.55
63.31
63.64
64.58
64.44
60.46
60.42
83.58
84.11
84.15
84.03
82.73
83.40
85.76
86.68

36-bit
59.93
60.56
64.05
64.80
60.85
84.97
83.52
82.93
87.08

(a)

(b)

(c)

(d)

(e)

(f)

48-bit
59.84
60.74
64.28
65.60
60.93
84.80
81.31
82.91
86.80

Fig. 3. The Precision-Recall curves computed by LSH, ITQ, DPLM, SDH, SGH, DPSH, ADSH, DAPH, and DADH on the MIRFLICKR-25K dataset.
Figures from (a) to (f) are associated with the code length 8-bit, 12-bit, 16-bit, 24-bit, 36-bit and 48-bit.

still reaches the best point. Except the case when the bit
length is 8, DADH always gain 4% or more enhancement in
MAP@Top500 and Precision@Top500 scores, indicating the
effectiveness of our method.
The Precision-Recall curves computed by different methods
on the IAPR TC-12 dataset are displayed in Fig.2, when the
bit length changes from 8 to 48. We can easily observe that
covered areas gained by DADH are much larger than that
obtained by other comparison methods. We can find that the
proposed method can dramatically outperform the traditional
data-independent and data-dependent strategies. Referring to
DPSH, ADSH and DAPH, there is also a better achievement
in all cases with different values of the code length.
2) MIRFLICKR-25K: The MAP results of the experiment
conducted on the MIRFLICKR-25K dataset are tabulated in

Tab.IV. We can see that DADH achieves the best performance
in all cases with different values of the code length. Being
similar to the results on IAPR TC-12 dataset, DPLM, SDH,
DPSH, ADSH, DAPH and DADH can obtain higher values
in MAP compared with LSH, ITQ and SGH. Referring to the
comparison between the traditional methods and deep learning
methods, DPSH, ADSH, DAPH and DADH dramatically
outperform LSH, ITQ, DPLM, SDH and SGH. Make a comparison between the proposed method with other deep hashing
approaches, DADH still has a more or less improvement. For
DADH, there is about 1.5%-3% enhancement on MAP scores
compared with these three deep hashing approaches.
The Top-500 MAP and Top-500 Precision scores on the
MIRFLICKR-25K dataset are displayed in Tab.V. It is easy
to observe that DADH obtains the best performance in both

JOURNAL OF IEEE TRANSACTIONS ON XX, VOL. X, NO. X, XX 2018

9

(a)

(b)

(c)

(d)

(e)

(f)

Fig. 4. The Precision-Recall curves computed by LSH, ITQ, DPLM, SDH, SGH, DPSH, ADSH, DAPH, and DADH on the CIFAR-10 dataset. Figures from
(a) to (f) are associated with the code length 8-bit, 12-bit, 16-bit, 24-bit, 36-bit and 48-bit.

TABLE VI
T HE MAP SCORES OBTAINED BY DIFFERENT METHODS ON THE
CIFAR-10 DATASET.

Method
LSH
ITQ
DPLM
SDH
SGH
DPSH
ADSH
DAPH
DADHL

8-bit
14.19
16.57
21.97
30.93
15.06
63.48
56.67
59.09
71.86

12-bit
13.26
17.12
22.76
32.50
15.43
66.97
71.41
61.17
75.12

16-bit
13.13
16.94
23.91
33.59
15.64
68.83
76.50
68.15
80.33

24-bit
13.84
17.08
25.89
35.36
16.07
73.45
80.40
69.22
81.70

36-bit
14.90
17.38
27.53
35.59
16.78
74.66
82.73
70.74
83.16

48-bit
15.13
17.58
28.85
36.53
16.88
75.02
82.73
70.28
83.90

MAP@Top500 and Precision@Top500, demonstrating the superiority compared with other existing strategies. With the
change of the code length, the MAP@Top500 and Precision@Top500 increase from (85.80%, 84.73%) to (87.58%,
86.80%), while the highest values obtained by LSH, ITQ and
SGH are only (62.04%, 60.93%), being much lower than ours’.
In contrast to DPLM and SDH, our strategy still gains more
than 20% enhancement in most cases. Furthermore, results
computed by DADH are much higher than that calculated by
DPSH, ADSH and DAPH. Concretely, scores of Top-500 MAP
and Top-500 Precision gained by DADH are almost always
higher than 85%, while these scores calculated by other deep
hashing methods are below than 85% in most cases.
Fig.3 show the Precision-Recall curves computed by differ-

ent methods on the MIRFLICKR-25K dataset, when the bit
length changes from 8 to 48. Note that we do not depict the
Precision-Recall curve obtained by LSH in Fig.3(f), since its
precision scores is far below than that of others. From Fig.3
we can observe that DADH remarkably outperforms LSH,
ITQ, DPLM, SDH, SGH, ADSH, and DAPH. Referring the
comparison between DPSH and DADH, the proposed method
is obviously superior to DPSH when the code length is 8 and
12, respectively. Although DPSH covers more areas when the
recall value is smaller than 0.4 in Fig.3(c)-(f), it is inferior
to DADH with the increase of the recall value. Overall, our
method still outperforms DPSH when the code length is 16,
24, 36 and 48.
3) CIFAR-10: Tab.VI lists the MAP scores obtained by the
proposed method and various comparison approaches on the
CIFAR-10 dataset. With the change of the code length from 8
to 48, the MAP scores computed by DADH rise from 71.86%
to 83.90%, being much higher than that obtained by traditional
hashing approaches, including LSH, ITQ, DPLM, SDH and
SGH. In contrast to DPSH and DAPH, it is easy to observe
that the presented method can achieve a better performance
under the different code length. Except the cases when code
length is 8 and 12, the MAP scores gained by DADH are
always higher than 80%, while the best performance computed
by DPSH and DAPH is only 75.02%. Also, ADSH is inferior
to the proposed method, especially when the code length is
small. This relatively indicates the effectiveness of our method
no matter the code length is small or large.

JOURNAL OF IEEE TRANSACTIONS ON XX, VOL. X, NO. X, XX 2018

10

TABLE VII
T HE T OP -500 MAP AND T OP -500 P RECISION SCORES OBTAINED BY DIFFERENT METHODS ON THE CIFAR-10 DATASET.

Evaluation
Method
LSH
ITQ
DPLM
SDH
SGH
DPSH
ADSH
DAPH
DADHL

8-bit
20.29
24.56
29.02
28.34
25.92
58.58
58.92
50.88
67.11

12-bit
19.92
27.72
34.07
34.78
25.11
66.85
70.07
66.24
73.08

MAP@Top500
16-bit 24-bit
19.81
21.17
27.55
28.65
35.66
38.86
37.92
41.69
25.91
27.13
71.48
75.74
74.95
78.09
72.43
77.31
78.42
82.02

36-bit
23.07
29.54
40.28
42.77
28.76
79.69
78.85
79.21
83.51

(a)

48-bit
24.42
30.24
41.67
44.57
29.46
80.62
77.61
80.40
84.17

8-bit
14.54
18.63
24.61
27.91
19.06
64.13
61.09
54.22
72.95

12-bit
15.26
21.88
29.72
33.83
20.10
70.80
73.71
68.28
77.53

Precision@Top500
16-bit 24-bit
16.19
18.13
22.53
24.59
31.93
35.62
37.31
42.39
21.53
23.46
74.71
78.34
78.85
81.84
74.42
77.36
82.68
84.18

(b)

36-bit
20.45
25.65
37.98
43.78
24.85
80.59
83.45
78.80
85.23

48-bit
21.44
26.40
39.84
45.08
25.42
81.55
82.86
79.55
85.59

(c)

Fig. 5. The MAP scores with the change of parameter τ , η and γ on three datasets, when the code length is 48.

(a)

(b)

(c)

Fig. 6. The change of objective function values and MAP scores with the increase of iterations.

The Top-500 MAP and Top-500 Precision scores computed
by different methods on the CIFAR-10 dataset are shown
in Tab.VII under the various code length. Obviously, four
deep hashing methods always achieve dramatic experimental results compared with rest traditional approaches. The
comparison between DADH and other deep hashing methods
also substantiates the superiority of the proposed strategy. In
contrast to DPSH, ADSH and DAPH, DADH has about 3%4% enhancement in both Top-500 MAP and Top-500 Precision
scores when the code length ranges from 12 to 48.
The Precision-Recall curves computed by different methods
on the CIFAR-10 dataset are depicted in Fig.4 under the bit
length ranging from 8 to 48. When the code length is 8, 12,

16, and 24, the Precision-Recall curves obtained by DADH
covers the most areas. Although the performance computed
by ADSH is competitive to ours’ when the code length is 36
and 48, DADH is still much superior to LSH, ITQ, DPLM,
SDH, SGH, DPSH, and DAPH.
E. Parameter Sensitivity Analysis
The MAP scores under the changes of different values of
τ , η and γ are shown in Fig.5. Note, we tune a parameter
with others fixed. For instance, we tune τ in the range of
[0.001, 1, 5, 10, 50, 100] by fixing η = 10 and γ = 100,
respectively. Similarly, we set τ = 10, γ = 100 in η tuning
and τ = 10, η = 10 in γ tuning. As we can see, our model is

JOURNAL OF IEEE TRANSACTIONS ON XX, VOL. X, NO. X, XX 2018

(a)

11

(b)

(c)

Fig. 7. The change of objective function values and MAP scores with the increase of iterations. Note that, the asymmetric terms tanh(F)BT − kS
2
tanh(G)BT − kS F are removed from the objective function.

insensitive to parameters. Specifically, τ , η and γ have a wide
range [1,50], [1,50] and [1,300], respectively. Our method
always achieves a satisfactory performance when τ , η and γ
are in these ranges. This relatively demonstrates the robustness
and effectiveness of the proposed method.

F. Convergence Analysis
To be honest, our proposed model can get a convergence
with a few of iterations. The change of the objective function
values and MAP scores on three datasets are displayed in
Fig.6 when the code length is 48-bit. It is easy to observe
that DADH converges to a stable value after less than 30
iterations. In fact, we further find that the asymmetric terms
2
2
tanh(F)BT − kS F and tanh(G)BT − kS F greatly contribute to the quick convergence. We try to remove these
two terms from our objective function to study the influence.
Note that, if the asymmetric terms are removed, the binary
code B are updated through sign(γ[tanh(G) + tanh(F)]).
2
As shown in Fig.7, if we remove tanh(F)BT − kS F and
2
tanh(G)BT − kS F from the objective function, not only
the MAP scores meet a degradation, but also our model
converges much slower compared with the original DADH,
indicating the necessity and significance of the asymmetric
terms.
V. C ONCLUSION
In this paper, we propose a novel deep hashing method
named dual asymmetric deep hashing learning (DADH) for
image retrieval. Specifically, two asymmetric networks are
designed to integrate the feature representation and hash function learning into the end-to-end framework. A pairwise loss
is introduced to exploit the semantic structure between each
pair outputs. Furthermore, another pairwise loss is proposed
to not only capture the similarity between the discrete binary
codes and learned real-value features, but also contribute to
a quick convergence at the training phase. Experiments are
conducted on three large-scale datasets and the outstanding
results substantiate the superiority of the proposed method.

2
F

and

ACKNOWLEDGMENT
The work is partially supported by the GRF fund from
the HKSAR Government, the central fund from Hong Kong
Polytechnic University, the NSFC fund (61332011, 61272292,
61271344, 61602540), Shenzhen Fundamental Research fund
(JCYJ20150403161923528, JCYJ20140508160910917), and
the Science and Technology Development Fund (FDCT) of
Macau 124/2014/A3.
R EFERENCES
[1] A. Andoni and P. Indyk, “Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions,” in Foundations of Computer
Science, 2006. FOCS’06. 47th Annual IEEE Symposium on. IEEE, 2006,
pp. 459–468.
[2] A. Gionis, P. Indyk, R. Motwani et al., “Similarity search in high
dimensions via hashing,” in VLDB, vol. 99, no. 6, 1999, pp. 518–529.
[3] B. Kulis and K. Grauman, “Kernelized locality-sensitive hashing for
scalable image search,” in Computer Vision, 2009 IEEE 12th International
Conference on. IEEE, 2009, pp. 2130–2137.
[4] Y. Gong, S. Lazebnik, A. Gordo, and F. Perronnin, “Iterative quantization:
A procrustean approach to learning binary codes for large-scale image retrieval,” IEEE Transactions on Pattern Analysis and Machine Intelligence,
vol. 35, no. 12, pp. 2916–2929, 2013.
[5] G. Lin, C. Shen, Q. Shi, A. Van den Hengel, and D. Suter, “Fast
supervised hashing with decision trees for high-dimensional data,” in
Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition, 2014, pp. 1963–1970.
[6] Y. Weiss, A. Torralba, and R. Fergus, “Spectral hashing,” in Advances in
neural information processing systems, 2009, pp. 1753–1760.
[7] W. Liu, J. Wang, S. Kumar, and S.-F. Chang, “Hashing with graphs,”
in Proceedings of the 28th international conference on machine learning
(ICML-11). Citeseer, 2011, pp. 1–8.
[8] W. Liu, C. Mu, S. Kumar, and S.-F. Chang, “Discrete graph hashing,”
in Advances in Neural Information Processing Systems, 2014, pp. 3419–
3427.
[9] W. Liu, J. Wang, R. Ji, Y.-G. Jiang, and S.-F. Chang, “Supervised hashing
with kernels,” in Computer Vision and Pattern Recognition (CVPR), 2012
IEEE Conference on. IEEE, 2012, pp. 2074–2081.
[10] F. Shen, C. Shen, W. Liu, and H. Tao Shen, “Supervised discrete
hashing,” in Proceedings of the IEEE Conference on Computer Vision
and Pattern Recognition, 2015, pp. 37–45.
[11] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification
with deep convolutional neural networks,” in Advances in neural information processing systems, 2012, pp. 1097–1105.
[12] K. Simonyan and A. Zisserman, “Very deep convolutional networks for
large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
[13] H. Liu, R. Wang, S. Shan, and X. Chen, “Deep supervised hashing for
fast image retrieval,” in Proceedings of the IEEE Conference on Computer
Vision and Pattern Recognition, 2016, pp. 2064–2072.

JOURNAL OF IEEE TRANSACTIONS ON XX, VOL. X, NO. X, XX 2018

[14] W.-J. Li, S. Wang, and W.-C. Kang, “Feature learning based deep supervised hashing with pairwise labels,” arXiv preprint arXiv:1511.03855,
2015.
[15] F. Shen, X. Gao, L. Liu, Y. Yang, and H. T. Shen, “Deep asymmetric
pairwise hashing,” 2017.
[16] C. Da, S. Xu, K. Ding, G. Meng, S. Xiang, and C. Pan, “Amvh: Asymmetric multi-valued hashing,” in Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition, 2017, pp. 736–744.
[17] Q.-Y. Jiang and W.-J. Li, “Asymmetric deep supervised hashing,” arXiv
preprint arXiv:1707.08325, 2017.
[18] M. Datar, N. Immorlica, P. Indyk, and V. S. Mirrokni, “Locality-sensitive
hashing scheme based on p-stable distributions,” in Proceedings of the
twentieth annual symposium on Computational geometry. ACM, 2004,
pp. 253–262.
[19] M. Raginsky and S. Lazebnik, “Locality-sensitive binary codes from
shift-invariant kernels,” in Advances in neural information processing
systems, 2009, pp. 1509–1517.
[20] Q.-Y. Jiang and W.-J. Li, “Scalable graph hashing with feature transformation.” in IJCAI, 2015, pp. 2248–2254.
[21] H. Jegou, M. Douze, and C. Schmid, “Product quantization for nearest
neighbor search,” IEEE transactions on pattern analysis and machine
intelligence, vol. 33, no. 1, pp. 117–128, 2011.
[22] F. Shen, C. Shen, Q. Shi, A. Van Den Hengel, and Z. Tang, “Inductive
hashing on manifolds,” in Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition, 2013, pp. 1562–1569.
[23] W. Kong and W.-J. Li, “Double-bit quantization for hashing.” in AAAI,
vol. 1, no. 2, 2012, p. 5.
[24] M. Norouzi and D. M. Blei, “Minimal loss hashing for compact binary
codes,” in Proceedings of the 28th international conference on machine
learning (ICML-11). Citeseer, 2011, pp. 353–360.
[25] P. Zhang, W. Zhang, W.-J. Li, and M. Guo, “Supervised hashing with
latent factor models,” in Proceedings of the 37th international ACM SIGIR
conference on Research & development in information retrieval. ACM,
2014, pp. 173–182.
[26] L. Fan, “Supervised binary hash code learning with jensen shannon
divergence,” in Proceedings of the IEEE International Conference on
Computer Vision, 2013, pp. 2616–2623.
[27] V. Erin Liong, J. Lu, G. Wang, P. Moulin, and J. Zhou, “Deep hashing for
compact binary codes learning,” in Proceedings of the IEEE Conference
on Computer Vision and Pattern Recognition, 2015, pp. 2475–2483.
[28] R. Xia, Y. Pan, H. Lai, C. Liu, and S. Yan, “Supervised hashing for
image retrieval via image representation learning.” in AAAI, vol. 1, 2014,
pp. 2156–2162.
[29] H. Lai, Y. Pan, Y. Liu, and S. Yan, “Simultaneous feature learning and
hash coding with deep neural networks,” in Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition, 2015, pp. 3270–
3278.
[30] Y. Cao, M. Long, J. Wang, and S. Liu, “Deep visual-semantic quantization for efficient image retrieval,” in CVPR, vol. 2, 2017, p. 6.
[31] K. Chatfield, K. Simonyan, A. Vedaldi, and A. Zisserman, “Return of the
devil in the details: Delving deep into convolutional nets,” arXiv preprint
arXiv:1405.3531, 2014.
[32] Q.-Y. Jiang and W.-J. Li, “Deep cross-modal hashing,” arXiv preprint
arXiv:1602.02255, 2016.
[33] H. J. Escalante, C. A. Hernández, J. A. Gonzalez, A. López-López,
M. Montes, E. F. Morales, L. E. Sucar, L. Villaseñor, and M. Grubinger,
“The segmented and annotated iapr tc-12 benchmark,” Computer Vision
and Image Understanding, vol. 114, no. 4, pp. 419–428, 2010.
[34] M. J. Huiskes and M. S. Lew, “The mir flickr retrieval evaluation,”
in Proceedings of the 1st ACM international conference on Multimedia
information retrieval. ACM, 2008, pp. 39–43.
[35] A. Krizhevsky and G. Hinton, “Learning multiple layers of features from
tiny images,” 2009.
[36] F. Shen, X. Zhou, Y. Yang, J. Song, H. T. Shen, and D. Tao, “A fast
optimization method for general binary code learning,” IEEE Transactions
on Image Processing, vol. 25, no. 12, pp. 5610–5621, 2016.
[37] A. Vedaldi and K. Lenc, “Matconvnet: Convolutional neural networks
for matlab,” in Proceedings of the 23rd ACM international conference on
Multimedia. ACM, 2015, pp. 689–692.

12

