Convolutional Hashing for Automated Scene Matching

arXiv:1802.03101v1 [] 9 Feb 2018

Martin Loncaric, Bowei Liu, Ryan Weber
Castle Global, Inc.
martin@thehive.ai
Abstract

frame pair

We present a powerful new loss function
and training scheme for learning binary hash
functions. In particular, we demonstrate our
method by creating for the first time a neural network that outperforms state-of-the-art
Haar wavelets and color layout descriptors
at the task of automated scene matching.
By accurately relating distance on the manifold of network outputs to distance in Hamming space, we achieve a 100-fold reduction
in nontrivial false positive rate and significantly higher true positive rate. We expect
our insights to provide large wins for hashing
models applied to other information retrieval
hashing tasks as well.

1

Introduction

error
type

benchmarks
with error

FN

all

FP

192-bit CLD

FP

64-bit
wavelets,
256-bit
wavelets

Figure 1: Select examples of false positives and false
negatives that state-of-the-art hashes made but our
64-bit CHASM correctly handled.

Many information retrieval tasks rely on high dimensional searches, including K-nearest neighbors
(KNN), approximate nearest neighbors (ANN), and
exact r-neighbor lookup in Hamming space. At
scale, these searches are enabled by indexes on binary
hashes, such as locality-sensitive hashing (LSH) and
multi-indexing [1]. Recent research has flourished on
these topics due to enormous growth in data volume
and industry applications [2]. We present a powerful new approach to a fundamental challenge in these
tasks: learning a good binary hash function.

1.1

Automated Scene Matching

ASM is an important information retrieval task, used
to perform reverse video lookup for broadcasting, research, and copyright infringement monitoring [3, 4].
The goal of ASM is to take a query sequence of video
frames and return all matching videos in a dataset,
along with the start and end times of the matches1 .

We demonstrate the effectiveness of our method by
applying it to the task of automated scene matching
(ASM) with a multi-index system. We call our model
convolutional hashing for automated scene matching
(CHASM). To the best of our knowledge, it is the first
neural network to outperform state-of-the-art hash
functions like Haar wavelets and color layout descriptors at ASM across the board.

For instance, suppose a research library indexes all
their documentaries for reverse video lookup. A researcher might then query the infamous Zapruder
film, and her results should be all the documentaries
containing a subset of it. The results should also include the specific time segments these documentaries
matched the Zapruder film.

1 ASM sometimes encompasses more than this specific definition. In particular, some ASM research aims to retrieve footage
of the same 3D scene based on videos taken from another angle [5].
2 Other procedures exist, especially ones that downsample to heuristic-selected keyframes rather than using a fixed frame

1

For large video datasets, this can be solved by imple- We compare our approach to state-of-the-art ASM
menting the following procedure2 :
methods, as well as variants of our own method,
trained with other binary hash loss functions from
1. For each video in the dataset:
recent research [6, 7].
(a) Downsample to a fixed frame rate (fps) and
image size.

1.2

(b) Create a binary hash of each frame.

Multi-Indexing

(c) Using a multi-index lookup table, index
Multi-indexing can enable search within a Hamming
each binary hash, pointing back to the
radius r by splitting the n-bit hash into r + 1 subsource video and timestamp (Section 1.2).
strings of length n/(r + 1) [1]3 . Each of these sub2. For each query video:
strings is inserted into its own index, pointing back
to the full hash, video, and timestamp.
(a) Apply the same fps and image size downsampling.
Lookup is performed as follows:
(b) Create a binary hash of each frame.

1. Taking an input hash h, split into substrings
h1 , . . . hr+1 .

(c) Retrieve matches from the index for each
binary hash.

2. Initialize an empty list L.

(d) Based on the individual frame matches,
use heuristics to decide which dataset
videos match during which time segments.

3. For i = 1, . . . r + 1, add exact matches for hi in
the ith index to L.
4. Filter duplicate results out of L.

Our work optimizes the binary hash function used
in steps 1b and 2b of this procedure. An ideal hash
function for ASM must satisfy many requirements:

5. Filter results with Hamming distance > r out
of L.

• Frames from the same video that are offset by
6. Return L
up to a small time difference t0 should map to
hashes within the Hamming radius r so that The expected lookup runtime scales with r + nm,
videos with a time shift still match together.
where m is the number of exact matches per sub• Frames offset by more time should map to string. Therefore, with CHASM we seek to minimize
hashes outside the Hamming radius so that the not only the overall false positive rate, but also the
matching heuristics can determine precise start false positive rate for each index.
and end times.
• Frames from different videos should map to
hashes outside the Hamming radius to avoid
false positive matches.

2

Related Work

• The false positive rate for each of the multi- 2.1 Learning Binary Hash Functions
index’s indices must be extremely low, since the
dataset may be very large, and each false posiRelevant to our method, some work has been done to
tive increases query time and the probability of
find a good general method for learning binary hash
mismatching scenes.
codes. Thus far these methods have relaxed discrete
It is worth noting that as dataset size increases, an Hamming distance losses into differentiable optimizaASM hash function’s precision and recall drop, but its tions by using piecewise-linear transformations on the
true positive and false positive rates stay the same. hash embeddings [8, 9]. In this work we take these
Therefore we used true positive and false positive ideas further and leverage a more natural transformarates as our test metrics.
tion.
rate. However, these approaches are plagued by low recall and cannot distinguish between time granularities finer than their
keyframes [3].
3 In scenarios with a combination of extremely large datasets, short hashes, and large r, it may be more practical to use
fewer than r + 1 substrings and make up for the missing Hamming radius with brute-force searches around each substring [1].
However, for ASM these conditions can be avoided by using larger hashes.

2

2.2

ASM

Occasionally other loss terms are applied, including
MSE from predefined target hash codes [19] and adSo far neural networks have failed to outperform versarial error [16].
hand-picked features at hashing for ASM. The main
We experimented heavily with these loss functions,
difference among existing state-of-the-art approaches
but ultimately developed our own. However, ideas
comes from their hash functions, which are typically
from CBVR and CBIR papers such as using loss
chosen from the frequency responses of some basis
terms between each pair of images in a batch [13]
[3]. For wavelets, the discrete wavelet transform is
proved valuable in creating a good training scheme
run on images in grayscale, returning embeddings in
for CHASM.
the corresponding basis [3]. The most common color
descriptor representation is Color Layout Descriptor Another notable trend in CBVR and CBIR research is
(CLD), which performs a discrete cosine transform the use of either binarization loss or learning by conon each channel of a smoothed image in YCbCr color tinuation [7]; that is, gradually sharpening sigmoids
space. Each of these embeddings is generally bina- to force embeddings close to ±1. This draws inspirarized with a 1 for each above-median response and a tion from the iterative quantization (ITQ) approach,
0 for the others [5].
which solves an alternating optimization problem of
improving the embedding based on other metrics,
The state of ASM research leaves a major gap:
then updating a rotation matrix to minimize binalearned methods that can perform temporally accurization loss [20]. Unlike ITQ, more recent papers
rate scene matching quickly on very large datasets. In
now allow gradients caused by binarization loss and
this paper, we used 3 benchmarks: the 64-bit (8x8)
learning by continuation to backpropagate through
Haar wavelet hash, the 256-bit (16x16) Haar wavelet
their network.
hash, and the 192-bit CLD hash.
We find that backpropagating binarization loss or
using learning by continuation causes learning to
2.3 CBVR and CBIR
plateau, with the model only learning from a shrinking gray area of data points in between disconnected
Content based video retrieval (CBVR) is a broad regions of data points near the corners of the hypertopic that involves using video, audio, and/or meta- cube {−1, 1}n . Moreover, the values in these regions
data to retrieve similar videos from a dataset. This do not binarize with the sign function any differently
is an easier task than ASM in that an entire video is than less extreme values. The main blocker preventretrieved, rather than a specific video segment. There ing researchers from abandoning these methods is
has been some recent research into learning a binary that Euclidean distance becomes a bad approximahash function for entire videos based on high-level, tion for Hamming distance otherwise.
semantic labels [10, 11].
Similarly, the objective of content based image retrieval (CBIR) is to take a query image as input and
return a set of similar images in an image dataset.
Many recent papers in this field have also used deep
learning approaches to train embeddings that get binarized into hashes.

3
3.1

Method
Interpretation of Embedding

We propose an alternative to binarization loss and
learning by continuation that respects the geometry
of the embedding without punishing intermediate val• terms that minimize or maximize the Euclidean ues. We instead let our model produce embeddings
distance between embeddings depending on following an approximately Gaussian distribution.
whether they belong to similar or dissimilar content [12, 13, 6, 14, 15, 7, 10, 11, 16]
3.1.1 Distribution of Embedding
• classification loss terms that use a bottleneck
before the classification layer as the hash layer
Let x(f ) = (x1 (f ), . . . xn (f )) be the vector of hash
[17, 18, 10, 11]
node outputs for an input frame f , and let F be the
• binarization loss terms that punish embeddings distribution of video frames to consider. We motivate
for being far from ±1 [13, 6, 14, 15, 16]
our loss function with the following assumptions:

So far, deep learning papers in these topics have
mainly used a combination of three loss terms:

3

• If f ∼ F is a random video frame variable, then
xi (f ) ∼ N (0, 1) (enforced by batch normalization of xi and a loss term on skew).
• xi is independent of other xj .
Let y(f ) = x(f )/||x(f )||2 be the L2 -normalized output vector. Since x(f ) is a vector of n independent
random normal variables, y(f ) is a random variable
distributed uniformly on the hypersphere.

yi
−y i

This L2 -normalization is the same as SphereNorm [21]
and very similar to Riemannian Batch Normalization
[22]. Liu et al. posed the question of why this technique works better in conjunction with batch norm
than either approach alone, and our work bridges that
gap. An L2 -normalized vector of IID random normal
variables forms a uniform distribution on a hypersphere, whereas most other distributions would not.
An uneven distribution would limit the regions on
the hypersphere where learning can happen and leave
room for internal covariate shift toward different, unknown regions of the hypersphere.
3.1.2

π

θ

yj

Figure 2: An arc of length θ on the unit hypersphere
starting from a random point in a random direction
has probability θ/π for the sign of a particular component to change along its course. In the 3D example
above, crossing the great circle implies that the sign
of one component differs between y i and y j .
Prior hashing research has made inroads with a similar observation, but applied it in the limited context
of choosing vectors to project an embedding onto for
binarization [23]. We apply this idea directly in network training.

Estimate of Distribution of Hamming
Distance

To avoid the assumption that Euclidean distance
translates to Hamming distance, we further study
the distribution of Hamming distance given these L2 normalized vectors. We derive the exact probabil- 3.2 Classes of Time Differences
ity that two bits match, given two uniformly random
points y i , y j on the hypersphere, conditioned on the For brevity, we define four classes of pairs of frames,
angle θ between them.
depending on how far separated in time they are (Tai
j
We know that y · y = cos(θ), so the arc length of ble 1). Our goal in CHASM is to maximize how often
the path on the unit hypersphere between them is frame pairs in H0 match together while minimizing
arccos(y i · y j ). A half loop around the unit hyper- how often frame pairs in H1 , H2 , and H3 do. Among
sphere would cross each of the n axis hyperplanes (i.e. H1 , H2 , and H3 , it is by far most important that
yk = 0) once, so a randomly positioned arc of length frame pairs in H3 do not match together, since by
θ crosses nθ/π axis hyperplanes on average. Each far most frames in a video index will be from videos
axis hyperplane crossed corresponds to a bit flipped, different than the query.
so the probability that a random bit differs between
these vectors is

arccos y i · y j
ij
P =
π

Table 1: Classes of frame pairs
name
H0
H1
H2
H3

time difference
≤ t0
> t0
> t0
∞

same shot
yes
yes
no
no

same video
yes
yes
yes
no

Given this exact probability, we estimate the distribution of Hamming distance by making the approximation that each bit position between the two vectors
differs independently from the others with probability
P ij . Therefore, the probability of Hamming distance
being within r is approximately F (r; n, P ij ) where 3.3 Loss Function
F is the binomial cumulative distribution function.
This approximation proves to be very close for large With batch size b, let X = (x1 , . . . xb )T be our batchn (Figure ??).
normalized logit layer for a batch of frames (f 1 , . . . f b )
4

105

exact binomial distribution
empirical true distribution

105

104

104
103

count

count

exact binomial distribution
empirical true distribution

103

102

102

101

101

100

100
0

2

4

6
8
10
Hamming distance

12

14

16

0

5

10

15
20
Hamming distance

25

30

Figure 3: The distribution of Hamming distance for two uniformly random vectors on the n-hypersphere, conditioned on being separated by an angle θ = 0.2. From left to right, n = 64, 192. Each empirical distribution
was calculated from the results of 106 trials.
and Y = (y 1 , . . . y b )T be the L2 -row-normalized version of X; that is, y i = xi /||xi ||2 . Similarly, let
n
Y1 , . . . Yr+1 be the b × r+1
L2 -row-normalized submatrices formed by splitting X into vertical slices;
in other words, define a submatrix Yl for the logarccos(Y T Y )
its of the lth substring. Let P =
and
π
arccos(YlT Yl )
Pl =
.Let W be the vector of all our
π
model’s learnable weights. Let U1 , U2 , U3 be b × b matrices that depend on which class each pair of frames
(f i , f j ) is in (Table 2). We define our loss to be

• J5 = ||W ||22 , a regularization term on the
model’s learnable weights to minimize overfitting. We used λ5 = 10−5 .
Table 2: Loss weights by frame pair class of (f i , f j )
Weight
U1ij
U2ij
U3ij

H0
1
0
0

H1
0
5
0

H2
0
5 × 102
102

H3
0
105
2 × 104

Note that terms J1 , J2 , and J3 work on all pairwise
combinations of images in the batch, providing us
with a very accurate estimate of the true gradient.

J = −J1 − J2 − J3 + λ4 J4 + λ5 J5
with

• J1 = Avg [U1 ◦ ln F (r; n, P )], the classweighted average log likelihood of each pair of 3.4 Dataset
frames to be within Hamming distance r (Table
We trained our model using frames from Google’s
2).
AVA video dataset [24], which consists of 154 training
• J2 = Avg [U2 ◦ ln F (n − r − 1; n, 1 − P )], the and 38 test videos annotated with activities. For our
class-weighted average log likelihood of each purpose of automated scene matching, we disregarded
pair of frames to be outside Hamming distance the activity annotations. We were able to obtain 136
r (Table 2).
of the training videos and 36 of the test videos.
Pl
• J3 =
j=1 Avg [U3 ◦ ln F (m − 1; m, 1 − Pj )], To ensure that our model would learn meaningful simthe class-weighted log likelihood that substrings ilarities between frames, we selected the distinct cutdiffer between frames, summed over each sub- free shots of each video. Then we filtered down to
string (Table 2). This term is particular to shots at least 4 seconds long and cut them to a maxmulti-indexing.
imum of 8 seconds. We used a subset such that each
was separated by at least 60 seconds from any other.
2
P
b
1
i 3
, penalizing high skew We then took training and testing shots from videos
• J4 = nb
i=1 (x )
2
and enforcing our assumption that the embed- in the respective category, downsampling each shot at
ding follows a Gaussian distribution. We used 15fps and 64 × 64 resolution to produce video frames.
λ4 = 2.
We used all frames from the training set in training
5

3.6

and distinct subsets from the test videos for validation and testing.

Training Scheme

We chose t0 to be 2/15, such that 2 frames sampled
To find the cuts in each video, we used a cut detec- at 15fps left or right of the query frame belong in H0 .
tion model defined by [25]. We will make our shot
Using a batch size of b = 280, we used what we call
annotations publicly available.
“hierarchical batches”, which include pairs of images
from each of H0 , H1 , H2 , and H3 . To construct one,
we

3.5

Architecture

• choose 35 random videos from our dataset with
replacement,
• choose 2 random shots from each video without
replacement,

The network that learns the hash function is composed of 5 main blocks of convolutions (Table 3).
Structurally it is similar to a wide resnet [26] with
the additional block added to handle the 64 × 64
input size. Additionally, the pooling, classification,
and softmax layers are removed and a fully-connected
layer is added to specify the hash size. By removing
the global pooling, we allow the network to learn information about the position of features in images,
which is important for automated scene matching.
We batch normalize the fully connected layer’s outputs, giving the embedding. From there, they are either L2 -normalized during training or binarized with
the sign function during inference.

• choose 3 random frames from each shot without
replacement, and
• choose 1 random additional frame within r for
each of those frames without replacement.
This ensures that even for very large datasets, each
class is available enough to train on.

We trained our model using stochastic gradient descent with momentum for 8M images, or s0 = 28, 600
steps. Our network’s weights randomly initialized to
configurations with very high J2 and J3 loss terms, so
we started our learning rate at a very low number β
Following [27] we used batch normalization before for the first 1000 steps, gradually increasing until we
each convolutional layer and remove the activation began a cosine decay at a more typical learning rate
function from the residual path. In all our experi- α:
ments, the depth factor was 6, which makes the net  s/1000
β α
work 49 convolutional layers. We experimented with
,
s < 1000
β
 
learning
rate
=
different width factors, but ultimately found no gains
α 1 + cos πs , s ≥ 1000
s0
for width factors over 1. This means the bulk of our
resnet is identical to that of [28].
where s is the global step.

To minimize overfitting, we used dropout with 30%
Table 3: Hash function architecture. Downsampling probability and flipped each batch of images horizonis performed by the first 3 × 3 convolution in each tally with 50% probability.
block with a stride of 2. Batch normalization and
ReLU activation precede each convolutional layer (except the first), and we add dropout between each con4 Results
volutional weight layer. We used r = 1 and N = 6.
group name
conv1

output size
64 × 64

conv2

64 × 64

conv3

32 × 32

conv4

16 × 16

conv5

8×8

fc

hash size

We trained CHASM models for hashes of 64 and 192
bits, optimizing for binary substrings size of 32. We
tested our results on the over 1.3 × 109 distinct pairs
of frames in our test set. CHASM achieved higher
true positive rates and lower false positive rates for
each class and hash size; in fact, even our 64-bit hash
beat the 192- and 256-bit benchmark hashes on both
true positive rate and H3 false positive rate by a large
margin (Figure 4).

block 
3
×
3, 16 × r

3 × 3, 16 × r
×N
3 × 3, 16 × r
3 × 3, 32 × r
×N
3 × 3, 32 × r
3 × 3, 64 × r
×N
 3 × 3, 64 × r 
3 × 3, 128 × r
×N
3 × 3, 128 × r


The lowest possible H3 false positive rate on our test
set was 4.71 × 10−6 , since various videos contained
6

Table 4: Positive rate by frame pair class and hash function. H0 and H3 are by far the most important
classes for these metrics. Values of r were chosen by scanning the ROC curves of true positive rate vs. H3
false positive rate for the best tradeoff (Figure 4).
Hash Function
Haar Wavelets (64-bit)
Haar Wavelets (256-bit)
CLD (192-bit)
CHASM (64-bit)
CHASM (192-bit)
CHASM-C (192-bit)
CHASM-B (192-bit)
CHASM-N (192-bit)

chosen r
3
14
16
3
7
8
1
7

H0 TP rate
0.810
0.834
0.835
0.885
0.8877
0.621
0.800
0.878

perfectly identical black frames. These results can be
avoided in practice by ignoring any perfectly black
frames, which are not very informative. Without
these frames, the 192-bit CHASM achieves 1.2 × 10−7
nontrivial H3 false positive rate at 88.8% true positive rate, over a 100-fold reduction of the 1.31 × 10−5
nontrivial H3 false positive rate of the best benchmark hash (256-bit wavelets) at 85.9% true positive
rate.

H1 FP rate
0.261
0.265
0.265
0.334
0.319
0.115
0.282
0.288

H2 FP rate
1.48 × 10−4
5.46 × 10−5
4.12 × 10−5
3.16 × 10−5
2.04 × 10−6
9.73 × 10−5
2.17 × 10−3
1.48 × 10−5

H3 FP rate
2.18 × 10−5
1.75 × 10−5
1.76 × 10−5
5.00 × 10−6
4.83 × 10−6
6.65 × 10−5
9.81 × 10−4
5.67 × 10−6

1.0

0.8

true positive rate

192-bit CHASM-B

In addition, we compare against two variants of
CHASM, modified by removing batch and L2 normalization on the embedding and using different loss
functions:

64-bit wavelets
256-bit wavelets
192-bit CLD
64-bit CHASM
192-bit CHASM
192-bit CHASM-C
theoretical optimum

0.6

0.4

0.2

• CHASM-B, using a squared error loss term between each frame pair depending on similarity
and an L1 binarization loss term for how far the
embedding is from binary.

0.0

0.0

0.5

1.0

1.5

2.0

H3 false positive rate

2.5

3.0

3.5
1e 5

Figure 4: True positive rate vs. false positive rate
among frames from different videos, plotted at different values of r. Each star corresponds to a heuristically chosen value of r on this dataset that has at least
60% TP rate and maximizes TP rate − 105 FP rate.
CHASM-N is omitted due to clutter.

• CHASM-C, using a logistic loss term based on
the dot product of embeddings. We also used
learning by continuation, computing the embeddings by passing our final layer through a tanh
layer that periodically gets sharper throughout
training.

We implemented the loss function from [6] for the
Conclusion
former and that of [7] for the latter, along with ap- 5
propriately tuned learning rate schedules and hyperparameters.
Our results show for the first time that a neural network is capable of outperforming traditional hashing
Neither approach performed on the same level as any methods at the task of hashing for ASM. Our model
of our benchmark hashes, let alone CHASM (Table was able to reduce nontrivial false-positive rate on a
4). However, this is not in contradiction with their large dataset by a factor of 100, even at higher true
respective papers’ results; both worked for low-recall, positive rate. This constitutes a massive improvement
high-precision tasks like finding nearest neighbors.
in the speed and accuracy of ASM systems.
In contrast, we found that state-of-the-art approaches
to CBIR were unable to beat even our benchmarks.
We attribute our model’s comparable success to four
main factors.

Finally, we compared against a model CHASM-N
trained without skew loss (J4 ). We found that it had
lower true positive rate and higher H3 false positive
rate for every value of r ≤ 16.
7

• CHASM’s loss depends on the chance of misclassifying an image.

2016 IEEE Conference on Computer Vision and
Pattern Recognition (CVPR), pages 2064–2072,
2016.

• CHASM uses good estimates for the distribution of Hamming distance as a function of embeddings.

[7] Zhangjie Cao, Mingsheng Long, and Philip S.
Yu. Hashnet: Deep learning to hash by continuation. arXiv:1702.00758 [], 2017.

• CHASM does not restrict the embedding’s values near ±1 during training, which (without actually changing its binarized values) can prevent
the model from learning.

[8] Mohammed Norouzi and David J. Fleet. Minimal loss hashing for compact binary codes. In
International Conferences on Machine Learning,
2011.

• CBIR research has focused on low-recall,
moderate-precision regimes like nearest neighbors, whereas ASM demands extremely low
false positive rate.

[9] Mohammed Norouzi, David J. Fleet, and Ruslan
Salakhutdinov. Hamming distance metric learning. In Conference on Neural Information Processing Systems, 2012.

We also shed light on why L2 -normalization of layer
outputs improves learning in conjunction with batch
norm. Perhaps most importantly, we provide a pow- [10] Yajiao Dong and Jianguo Li. Video retrieval
erful new loss function and training scheme for learnbased on deep convolutional neural network.
ing binary hash functions in general.
arXiv:1712.00133 [], 2017.
[11] Xu Liu, Lili Zhao, Dajun Ding, and Yajiao Dong.
Deep hashing with category mask for fast video
retrieval. arXiv:1712.08315 [], 2017.

References

[1] Mohammad Norouzi, Ali Punjani, and David J.
[12] Rongkai Xia, Yan Pan, Hanjiang Lai, Cong Liu,
Fleet. Fast search in hamming space with multiand Shuicheng Yan. Supervised hashing for imindex hashing. In IEEE Conference on Computer
age retrieval via image representation learning.
Vision and Pattern Recognition, pages 3108–
In AAAI Conference on Artificial Intelligence,
3115, 2012.
2014.
[2] J. Wang, W. Liu, S. Kumar, and S. F. Chang.
Learning to hash for indexing big data?a survey. [13] Venice Erin Liong, Jiwen Lu, Gang Wang, Pierre
Moulin, , and Jie Zhou. Deep hashing for comIn IEEE Transactions on Pattern Analysis and
pact binary codes learning. Conference on ComMachine Intelligence, volume 104(1), pages 34–
puter Vision and Pattern Recognition (CVPR),
57, 2016.
2015.
[3] Aasif Ansari and Muzammil Mohammed. Content based video retrieval systems - methods, [14] Han Zhu, Mingsheng Long, Jianmin Wang, and
Yue Cao. Deep hashing network for efficient simtechniques, trends and challenges. In Internailarity retrieval. AAAI, 2016.
tional Journal of Computer Applications, volume
112(7), 2015.
[15] Wu-Jun Li, Sheng Wang, and Wang-Cheng
Kang. Feature learning based deep supervised
[4] Markus Mühling, Manja Meister, Nikolaus Kohashing with pairwise labels. In International
rfhage, Jörg Wehling, Angelika Hörth, Ralph
Joint Conference on Artificial Intelligence, 2016.
Ewerth, and Bernd Freisleben. Content-based
video retrieval in historical collections of the german broadcasting archive. In International Con- [16] Zhangjie Cao, Mingsheng Long, Chao Huang,
and Jianmin Wang. Transfer adversarial hashing
ference on Theory and Practice of Digital Lifor hamming space retrieval. arXiv:1712.04616
braries, pages 67–78, 2016.
[], 2017.
[5] B V Patel and B B Meshram. Content based
video retrieval systems. International Journal of [17] Kevin Lin, Huei-Fang Yang, Jen-Hao Hsiao,
and Chu-Song Chen. Deep learning of binary
UbiComp, 3(2), 2012.
hash codes for fast image retrieval. Conference
[6] H. Liu, R. Wang, S. Shan, and X. Chen. Deep
on Computer Vision and Pattern Recognition
supervised hashing for fast image retrieval. In
Workshops (CVPRW), 2015.
8

[18] Huei-Fang Yang, Kevin Lin, and Chu-Song
Chen.
Supervised learning of semanticspreserving hash via deep convolutional neural
networks. IEEE Transactions on Pattern Anal[24]
ysis and Machine Learning, 2017.
[19] Xuchao Lu, Li Song, Rong Xie, Xiaokang Yang,
and Wenjun Zhang. Deep binary representation
for efficient image retrieval. Advances in Multimedia, 2017.

sensitive hashing. In Conference on Neural
Information Processing Systems, pages 108–116,
2012.
Chunhui Gu, Chen Sun, David A. Ross, Carl
Vondrick, Caroline Pantofaru, Yeqing Li, Sudheendra Vijayanarasimhan, George Toderici,
Susanna Ricco, Rahul Sukthankar, Cordelia
Schmid, and Jitendra Malik. Ava: A video
dataset of spatio-temporally localized atomic visual actions. arXiv:1705.08421 [], 2017.

[20] Yunchao Gong and Svetlana Lazebnik. Iterative
quantization: A procrustean approach to learn- [25] Michael Gygli. Ridiculously fast shot bounding binary codes. IEEE Transactions on Pattern
ary detection with fully convolutional neural netAnalysis and Machine Intelligence, 35(12):2916–
works. arXiv:1705.08214, 2017.
2929, 2013.
[26] Sergey Zagoruyko and Nikos Komodakis. Wide
[21] Weiyang Liu, Yan-Ming Zhang, Xingguo Li,
residual networks. arXiv:1605.07146 [],
Zhiding Yu, Bo Dai, Tuo Zhao, and Le Song.
2016.
Deep hyperspherical learning.
In Advances
in Neural Information Processing Systems 30 [27] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and
(NIPS 2017) pre-proceedings, 2017.
Jian Sun. Identity mappings in deep residual networks. In ECCV, 2016.
[22] Minhyung Cho and Jaehyung Lee. Riemannian
approach to batch normalization. In Advances [28] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and
in Neural Information Processing Systems 30
Jian Sun. Deep residual learning for image recog(NIPS 2017) pre-proceedings, 2017.
nition. 2016 IEEE Conference on Computer
Vision and Pattern Recognition (CVPR), pages
[23] Jianqiu Ji, Jianmin Li, Shuicheng Yan,
770–778, 2016.
Bo Zhang, and Qi Tian. Super-bit locality-

9

