arXiv:1702.06383v2 [cs.IR] 12 Nov 2017

Differential Geometric Retrieval of Deep Features
Y Qian and E Vazquez

B Sengupta

Cortexica Vision Systems Limited
30 Stamford Street SE1 9LQ
London, UK
yu.qian@cortexica.com

Cortexica Vision Systems Limited
Imperial College London
London, UK
b.sengupta@imperial.ac.uk

Abstract—Comparing images to recommend items from an
image-inventory is a subject of continued interest. Added with
the scalability of deep-learning architectures the once ‘manual’
job of hand-crafting features have been largely alleviated,
and images can be compared according to features generated
from a deep convolutional neural network. In this paper, we
compare distance metrics (and divergences) to rank features
generated from a neural network, for content-based image
retrieval. Specifically, after modelling individual images using
approximations of mixture models or sparse covariance estimators, we resort to their information-theoretic and Riemann
geometric comparisons. We show that using approximations
of mixture models enable us to compute a distance measure
based on the Wasserstein metric that requires less effort
than other computationally intensive optimal transport plans;
finally, an affine invariant metric is used to compare the
optimal transport metric to its Riemann geometric counterpart
– we conclude that although expensive, retrieval metric based
on Wasserstein geometry is more suitable than information
theoretic comparison of images. In short, we combine GPU
scalability in learning deep feature vectors with statistically
efficient metrics that we foresee being utilised in a commercial
setting.

1. Introduction
A common problem in computer vision lies in finding
similarity between 2 (or 3)-dimensional images (or tensors).
This is attained by measuring distances between the two objects, primarily using normalised co-relation, Euclidean distance, Bhattacharyya distance, Jensen-Shannon divergence,
amongst many others. The distances are measured after the
images are encoded in some latent space wherein such a
latent structure is learnt using a variety of classifiers –
support vector machines (SVMs), logistic regression, etc.
Recently, due to the advantages of scalability, large-scale
classifier frameworks based on deep-learning have been
used for music recommendation [29], image recommendation [25] as well as general recommendation architectures.
[5]. Most of these frameworks do not take the underlying
YQ and BS contributed equally to this manuscript

geometry of the feature space into account while making
a recommendation. This becomes increasingly important
when the similarity between objects is measured in terms
of ‘perception’, a quantity that is oblivious to the commonly used distance metrics. The non-trivial problem lies
in (a) collecting perceptual similarity between objects in a
database (via psychophysics), and (b) using this similarity
to construct a metric for classification and retrieval (metric
learning, learning to rank, etc.). To compound the problem
further, metric used for comparing images might be very
different than those for comparing sounds. Regardless of the
modality, a large stream of work in neuroscience hypothesise
that perception is based on minimising the prediction error
between what is observed and what we predict we will
observe [11].
In this paper, we start with the 0th order problem, i.e.,
compare how different distance metrics fair against one
another when objects that are to be compared are represented
as feature spaces induced by a deep neural network; furthermore, we use approximations of the probability density to
compute a metric based on the principle of optimal transport
[30] and Riemann geometry [1] that takes into account the
geometry of transport between two images, an idea that
is inherently essential for solving the perceptual similarity
problem.
Technically, the problem lies in searching an image database (ranking) with millions of image features
(Ani ×rm ×fj ) for sets of images (say up to 10-20 images)
that have similar properties to a query image (âquery ).
Here, A is a ni × rm × fj tensor where ni is the total
number of images, rm is the length of each feature vector
and fj is the total number of features extracted from one
image; âquery is a rm × fj query matrix. Although, one
can manually construct feature-vectors based on wavelet
decomposition, low-rank approximations, etc., we rely on
using a convolution neural network (CNN) to compute the
feature signature (rm × fj ) for images in the database, as
well as the query.
One way to operationalise a solution lies in weighting
each image in the database using a weight vector, and
subsequently, extremise the mutual information (or another
comparison metric) between the query and the database
with respect to the weights. The result of this optimisation

problem leads us to a weight vector that provides a rank for
all the images in the database when compared to the query
image. This is equivalent to measuring distances where each
image lies on a continuous probability manifold. There are
two contributions of this paper – (a) in order to describe
each image with its deep feature, we use either a computationally efficient approximation of Gaussian Mixture
Models (GMMs) or a sparse covariance estimator based on
Given rotations, and (b) we provide comparison between
these probability distributions using a variety of informationtheoretic and geometric metrics. This work leads us to a
much deeper problem where geometric similarity measures
can be possibly combined to approximate the metric governing perceptual similarity.

subsection 2.2.1-2.2.3, we describe three approaches to rank
images in terms of their ‘statistical similarity’ (not perceptual similarity). For the first, we use an information-theoretic
divergence while the second and third distances are based on
the cost involved in transporting one image to another, and
geodesic distance on a Riemannian manifold, respectively.
To rank images in the database we use two methods,
one is to build a Gaussian Mixture Model (GMM) [20], and
the second is to estimate a covariance matrix from deep
features. For each image, we model the rm × fj feature
matrix using a GMM. Specifically, for computational and
analytical efficiency (baseline measure), we approximate the
GMM with a Normal distribution, such that the sufficient
statistics read,

2. Methods
µ̃ =

2.1. Dataset and deep-feature generation
In this paper, Describable Textures Dataset (DTD) [6]
is used to evaluate geometric similarity measures for image
retrieval. Images in DTD are collected from wild images
(Google and Flickr) and classified based on human visual
perception [28], such as directionality (line-like), regularity
(polka-dotted and chequered), etc. DTD is therefore selected
in this research to evaluate the similarity measurements
base on human visual perception. DTD contains 5640 wild
texture images with 47 describable attributes drawn from
the psychological literature and is publicly available on the
web at http://www.robots.ox.ac.uk/vgg/data/dtd/.
Textures can be described via orderless pooling of filter
bank response [12]. In Deep CNN, the convolutional layers
are akin to non-linear filter banks; these have in fact been
proved to be better for texture descriptions [7]. Here, the
deep local features are extracted from last convolutional
layer of a pre-trained VGG-M [4].
 This is represented by
A = a1 , ..., ai , ..., aN : a ∈ RD ; the size of last convolutional layer is H × W × D, where D denotes the dimension
vector of filter response at the ith pixel of last convolution
layer; N = H × W is the total number of local features.
For image level representation, two methods are applied
to local features – one is to generate a Gaussian Mixture
Model (GMM) model on local descriptors and the second
is to estimate a shrunk yet sparse co-variance matrix from
the deep feature representation of individual images. A
statistical similarity metric is then applied to rank images.
As a baseline for distance p
calculations, we compute Eux21 + . . . + x2n ) between the
clidean distances (kxk2 =
query image and the database. Similarly, to establish a
baseline for feature extraction, we use the Bag of Words
(BoW) composed of scale-invariant feature transform (SIFT)
features. For further details on SIFT and BoW, please refer
to [25].

2.2. Retrieval and Ranking
Image retrieval using Euclidean norm with bag-of-words
feature encoding has been described elsewhere [25]. In

X

ωa µa

a

Σ̃

=

X



T
ωa Σa + (µa − µ̃)(µa − µ̃)

(1)

a

µa , Σa and ωa are the mean, co-variance and the mixing
weights of each Normal distribution (subscript a).
The second approximation to an image relies on estimating the co-variance matrix from the feature matrix generated
from a deep convolutional neural network. Although the geometry of the co-variance matrix can be utilised to estimate
it using low-rank and sparse penalisation, for the sake of
computational efficiency, we use an alternative treatment
due to [2], [3], i.e., a fast sparse matrix transformation
(SMT). Briefly, the SMT imposes sparsity constraint on the
manifold of co-variance matrices yet maintains a full-rank
representation. This is useful as the computation is O(fj );
the SMT can also be seen as a generalisation of FFT and
orthonormal para-unitary wavelet transform.
We will assume that each feature vector is i.i.d zero
mean Normal random vectors, and the sample covariance is
simply, n1 AAT ; it is a unbiased estimate of the true covariance matrix, R = E[S] = EΛE T . Often time S is singular,
and shrinkage estimators [15] are used to regularise the
covariance matrix by shrinking it towards a target structure
such as an identity matrix, a diagonal matrix with sample
variances, amongst others. Sparsity can also be imposed, as
in Graphical Lasso [10] by imposing a 1-norm constraint
on the precision matrix. The maximum likelihood (ML)
estimate of the eigenvectors (E ) and the eigenvalues (Λ)
give us,

Ê

=

Λ̂

=



arg min diag E T SE
E∈Ωk


diag Ê T S Ê

(2)

The SMT constrains the feasible set of Ωk to a set of
orthonormal transformations that are selected as an SMT of
order K. A matrix E is an SMT of order K if it can be
factorised to K sparse orthonormal matrices, i.e.,

E

=

K
Y

DKL (Va k Vb )
EVa [Vb ]

Ek =E1 E1 . . . Ek

k=1

Ek

=

I + Θ(ik , ij , θk )

(3)

Each sparse matrix E can be constructed as a orthonormal Givens rotation on a pair of co-ordinate indexes (ik , ij )
of Givens rotations such that,

cos(θk ) − 1, if i = j = ik or i = j = jk



 sin(θ ), if i = i and j = j
k
k
k
[Θ]ij =

−
sin(θ
),
if
i
=
j
and
j
=
ik
k
k



0, otherwise
Using greedy minimization [2], [3] we have,

EVa [Va ] − EVa [Vb ]
Va log Vb
X Z
X
ωb Vb
=
ωa Va log
φb|a
φb|a
a
b
X Z
X
ωb Vb
>
ωa Va
φb|a log
φb|a
a
b


 Z

X X
ωb
=
ωa
φb|a log
+ Va log Vb
φb|a
a
=
=

b

Here, φb|a is a variational parameter that is positive and
sums to one. Maximizing w.r.t φb|a yields,
EVa [Vb ] >

X

ωa log

a

(4)

=

arg min diag EkT Sk Ek

Sk+1

=

ÊkT Sk Êk
K
Y



ωa e−DKL (Va kVb ) −

X

ωa H (Va )

a

b

H is the entropy functional. Subsequently, the variational
bound becomes,
ωa0 e−DKL (Va kVa0 )
ωa log P
(10)
ωb e−DKL (Va kVb )

P

variational
DKL
(Va k Vb ) =

X
a

Êk

X

a0

b

var
We symmetrize the variational KL by using DKL
=
var iational
(Va kVb ) + 12 DKL
(Vb kVa ). Note that
such a divergence is the difference of two variational approximations, not a bound in itself.
1 var iational
2 DKL

2.2.2. Ranking via Kantorovich relaxation. Let (Ψ, ψm )
and (Λ, λm ) denote two Polish probability spaces depicting
k=1
image 1 and image 2, respectively – ψm and λm . The trivial
Λ̂ = diag (Sk+1 )
(5)
coupling between the two exists if Ψ and Λ are independent
so that the coupling is simply a tensor product ψm ⊗ λm .
As a final step, we obtain a shrunk co-variance matrix
A more useful coupling exists when there is a function
where the shrinkage parameter α is selected using crossS : Ψ → Λ such that λ = S(ψ). The transport map S
validation,
is equivalently the change of variables from ψm to λm .
Definition of a transport map: Let S be a Borel map: Ψ →
Λ, the push forward of ψm through S is the Borel measure,
denoted S#ψm defined on Λ by S#ψm (Λ) = ψm (S −1 (Λ)).
ΣSM T = Ê Λ̂Ê T
A Borel map: Ψ → Λ is said to be a transport map if
Σ = α · ΣSM T + (1 − α) · S
(6)
S#ψm = λm .
In optimal transport [30], there is a cost entailed by transporting one measure into another. The
2.2.1. Ranking by KL-divergence. Since there is no anatransport
map then relies on finding
the infimum of
lytical solution for the KL-divergence between two GMMs

R
c
(x,
S(x))
dψ
:
S
=
λ
.
Optimal
transference
m
#ψ
m
(Va ∼ Na (ωa , µa , Σa ) and (Vb ∼ Nb (ωb , µb , Σb ), we utilize
m
Ψ
plans are important because such couplings are stable to
two approximations: in the first we approximate the GMM
perturbations, they encode geometric information about the
with Eqn. 1. The KL-divergence (DKL (Va kVb )) now reads,
underlying cost-function, and they exist in smooth as well as



1
|Σb |
T −1
non-smooth settings. Given that the existence of this translog
− Nd + tr Σ−1
Σ
+
(µ
−
µ
)
Σ
(µ
−
µ
)
(7)
a
b
a
b
a
b
b
2
|Σa |
port map can not be guaranteed, a Kantorovich relaxation
In our experiments, we compute a symmetric-KL diamounts to a convex relaxation of Monge’s formulation
1
N ormal
vergence which is simply DKL
= 2 DKL (Va kVb ) +
wherein we seek a coupling γ ∈ P (Ψ, Λ),
1
2 DKL (Vb kVa ). Sorting the KL-divergence provides us
with a similarity rank.
Z

This is a gross-approximation wherein a more subtle
γ0 = arg min
c xψ , xλ dγ
(11)
γ∈P (Ψ,Λ)
approximation relies in bounding the KL-divergence. ParticΨ×Λ
ularly, using results from information theory [14], [21], we
provide retrieval results using a variational approximation
The joint probability measure with the marginals ψm
and λm allow us to define a Wasserstein distance of order
to the KL divergence. Particularly, since the log-function is
concave, using Jensen’s inequality we have,
p between ψm and λm ,
Ê

=

Êk

(8)

(9)

W (ψm , λm ) = inf

n

E D xψ , xλ

p o1/p

MAP
GMM-Normal KL
GMM-Variational KL
GMM-Wasserstein
SMT-Riemannian
SMT-Normal KL
SMT-Wasserstein
SIFT-BoW-Euclidean


(12)

D is a distance with a corresponding cost of
p
c xψ , xλ = d xψ , xλ . This Earth-Mover or the MongeKantorovich distance provides us with a metric over the
space of squared integrable probability measures. For two
Normal distribution, the L2 -Wasserstein distance [27], [30]
reads,

Top-1
0.53
0.45
0.62
0.50
0.53
0.59
0.43

Top-5
0.46
0.42
0.52
0.44
0.44
0.51
0.37

Top-10
0.42
0.38
0.46
0.39
0.39
0.46
0.32

Time
0.375s
0.016s
5.147s
0.754s
0.125s
9.04s
0.0007s

TABLE 1: Retrieval results on the DTD dataset. Note that
VGG-M has been pre-trained on Imagenet.
Precision curve for DTD dataset



1/2 
2
1/2
DW = kµa − µb k2 + tr Σa + Σb − 2 Σ1/2
Σ
Σ
(13)
b a
a

0.65
SIFT-Bow-Euclidean
GMM-Normal KL
GMM-Variational KL
GMM-Wasserstein
SMT-Riemannian
SMT-Normal KL
SMT-Wasserstein

Once the GMMs have been approximated via Eqn. 1 or
Eqn. 6, it is fairly simple to compute distances using Eqn.
13.
2.2.3. Ranking via Affine Invariant Riemannian Metric.
Let us again consider two feature matrices (query and
database), Va ∼ N (0, Σa ) and Vb ∼ N (0, Σb ). These
f ×f
positive-definite matrices are elements of S++
, a space
with a defined Riemannian metric [9], [19]. Under such
a geometry, the distance DR (Va , Vb ) between these two
matrices is,
DR (Σa , Σb ) =

=
F

C
X

2

log λc

0.55
0.5
0.45
0.4
0.35
0.3

#1/2

"
log(Σ−1/2
Σb Σa−1/2 )
a

Mean Precision for 94 queries

0.6

(14)

c=1

C is the dimension of the co-variance matrix, λc are the
eigenvalues, and F represents the Frobenius norm. A useful
property of such a distance is that regardless of how the
images are manipulated – be it re-scaling, normalisation,
whitening, filtering, etc. – the distance between the two
sources as captured by Eqn. 14 remains invariant.
To compute Eqn. 14 one can use Eqn. 1 to approximate both GMMs as Normal distributions; alternatively, the
covariance estimated using Eqn. 6 can be used.

3. Experiments
In this section, deep feature geometric retrieval methods
are evaluated on the DTD dataset. For each image, a set
of deep local features is extracted from last convolutional
layer of a pre-trained VGG-M. The dimension of each
local feature vector is 512. A GMM with 64 components
is subsequently generated from the set of local deep features. Normal approximation by GMM and sparse covariance estimation by SMT are used to represent the feature matrix; information-theoretic (Normal and Variational
approximation KL) and geometric (Wasserstein and Riemannian) measures to gauge the similarity of two images.
In this experiment, Normal approximation KL, Variational
approximation KL and Wasserstein metric is applied on
GMM model respectively and represented by GMM-Normal
KL, GMM-Variational KL and GMM-Wasserstein. Normal
approximation KL, Wasserstein and Riemannian metric are
applied on sparse covariance generated by SMT respectively

0.25
1

2

3

4

5

6

7

8

9

Rank

Figure 1: Precision on the DTD dataset

and denoted by SMT-Normal KL, SMT-Wasserstein and
SMT-Riemannian.
To evaluate similarity metric for image retrieval, Mean
Average Precision (MAP) on top 10 rankings are calculated. 2 images per category, i.e., a total of 94 images
are selected as queries from the test dataset. The dataset
retrieved includes 3760 images from DTD training and
validation datasets. MAP on DTD is listed in Table 1.

a)

b)

c)

d)

e)

f)

g)

Figure 2: Retrieved results on DTD: a) GMM-Wasserstein
b) SMT-Wasserstein c) GMM-Normal KL d) SMT-Normal
KL e) SMT-Riemannian f) GMM-Variational KL g) SIFTBoW Euclidean

10

Average precision on top 10 ranking is displayed in Figure
1. An example of the retrieval obtained with each method is
shown in Figure 2. On each case, ten images are displayed.
Top left image is the query used. The rest of images are
arranged by similarity to query image as obtained with each
method.
As is apparent from Table 1, computing Wasserstein
distances (using GMM or shrank co-variances) prove to be
superior when compared to the other methods evaluated,
including the baseline that uses a bag of words based SIFT
feature generation and Euclidean distances. Surprisingly,
variational KL is the least precise distance metric even
in contrast to the case where we approximate the feature
matrix with a univariate Normal distribution. A possible
cause for such a lower precision might be due to the inferior
conditioning of the co-variance matrix.
Query times are shown in Table 1; they have been
obtained as the average time to calculate the similarity
between two images. The code was implemented in Matlab
2015a under Linux with Intel(R) Xeon(R) CPU E5-2640 @
2.00GHz and 125G RAM.

4. Conclusion
Using hand-crafted features such as scale-invariant feature transform (SIFT), a histogram of oriented gradients
(HoG), etc. with Euclidean distances has had a long standing
history in computer vision, especially before the advent of
deep-learning based feature extractors. Hand-crafted features have poor generalisation capabilities along with being
non-robust to non-linear transformations. The same goes for
Euclidean distance, which is often not the ambient geometry
for the objects being compared. For probability measures,
the notion of an ambient geometry is clear due to the Riemann manifold inherited by these measures. In this paper,
we have touched upon the 0th order problem that may lead
to understanding ‘perceptual similarity’. More specifically,
we have used a convolution neural network (CNN) to obtain
feature matrices; utilising either Gaussian Mixture Models
(GMMs) or shrunk covariance estimators to obtain a probabilistic representation of the features. Subsequently, using
information theoretic divergences and Riemann geometric
metrics, we compare (dis) similarities between images.
Based on evaluation for DTD dataset, Wasserstein distances show increased retrieval fidelity based on both probabilistic representations, albeit they are more expensive to
evaluate. We believe that the increased accuracy of the
Wasserstein distance is due to two properties – first, the
metric does not include calculating the inverse of covariance matrices, thereby enclosing the cases with singularity;
in contrast, the KL-divergence between two distributions
could easily reach infinity if the covariance of the second
distribution becomes singular. The second property, which
we hypothesise, is the increased statistical robustness of the
Wasserstein distance, i.e., the metric might have small variance when comparing distributions that are closely situated
in the parametric manifold.

Although, we have utilised the final convolutional layer
of a CNN to distinguish images; much empirical work has
shown that there are many general features of an image or
a video that are captured by the initial layer of a CNN [31].
By visualising different layers in [18], it is apparent that the
lower layer of CNN can capture more colour information,
the higher layers, on the other hand, are more objective.
The retrieval result in Figure 2 demonstrates that colour is
not adequately captured due to deep local features extracted
from the last convolutional layer, which keeps less colour
related information. Hence, the fidelity to distinguish images using any of our retrieval criteria should undoubtedly
increase with additional ‘independent’ feature vector that
can be computed via the initial or the middle (general to
a more specific characterisation of the image) layers of a
CNN. Bayesian model averaging or multi-kernel learning, as
has been utilised for video-based action recognition might
be a way forward [23], [24].
Factors that affect the successful deployment

For a commercial system, speed is an essential ingredient. In fact, computing Wasserstein and Riemann distance
have their issues. For example, Wasserstein distance in
computer vision was proposed more than a decade ago
[22]. The cost of computing optimal transport between two
distributions of dimension d is at least O(d3 log d). This
is especially not plausible to compute in a commercial
environment when feature vectors are generated by deep
convolutional neural networks, which are by construction
high dimensional. In our study, even after approximating
the GMMs as multivariate Normal distributions, the computational inefficiency is inherent, as computing Eqn. 13
proves to be most expensive amongst all the metrics that
we compare. A solution emerges in the form of low dimensional embedding of the metric space [13], [16]; such
solutions introduce distortions in addition to an increase in
computational cost when the embedding dimension becomes
larger than four [8]. Additionally, they are not designed to
be scalable to take advantages of large-scale GPU resources.
[8] has suggested improving the scalability of the distance
calculation by using an iterative diagonal scaling algorithm,
known as Sinkhorn’s algorithm or iterative proportional
fitting. We leave this scalability issue for future work.
Similarly, computing the geodesic distance between two
co-variance matrices is equally time inefficient – O(4d3 ).
The main component of this inefficiency emerges from the
generalised eigenvalue equation, particularly for calculating
multiple Cholesky factorisations each time a query is initiated. One way forward may be to use Stein’s distance [26]
while preserving affine invariance and geometric properties
inherited by the covariance matrices. Another way ahead is
to perform the factorisation on a GPU [17]. This becomes
increasingly important if our framework were to be used
for indexing of videos (instead of images). This future
application relies on returning a set of similar videos in
response to a query video. This could replace the current

text based tagged video framework, like that used by several
online video platforms, with feature based tagged videos.

References
[1]
[2]
[3]
[4]
[5]

[6]
[7]
[8]
[9]
[10]
[11]
[12]

[13]
[14]

[15]
[16]
[17]
[18]
[19]
[20]
[21]

S. Amari, H. Nagaoka, and D. Harada. Methods of Information
Geometry. Translations of Mathematical Monographs. American
Mathematical Society, 2007. 1
G. Cao, L. R. Bachega, and C. A. Bouman. The sparse matrix
transform for covariance estimation and analysis of high dimensional
signals. IEEE Trans. Image Processing, 20(3):625–640, 2011. 2, 3
G. Cao, C. A. Bouman, and K. J. Webb. Noniterative MAP reconstruction using sparse matrix representations. IEEE Trans. Image
Processing, 18(9):2085–2099, 2009. 2, 3
K. Chatfield, K. Simonyan, A. Vedaldi, and A. Zisserman. Return
of the devil in the details: Delving deep into convolutional nets. In
British Machine Vision Conference, 2014. 2
H.-T. Cheng, L. Koc, J. Harmsen, V. Jain, X. Liu, and H. Shah.
Wide & deep learning for recommender systems. In Proceedings of
the 1st Workshop on Deep Learning for Recommender Systems, Deep
Learning for Recommender Systems 2016, pages 7–10, New York,
NY, USA, 2016. ACM. 1
M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, and A. Vedaldi. Describing textures in the wild. In Proceedings of the IEEE Conference
on Computer Vision and Pattern Recognition, 2014. 2
M. Cimpoi, S. Maji, and A. Vedaldi. Deep filter banks for texture
recognition and segmentation. In IEEE Conference on Computer
Vision and Pattern Recognition, 2015. 2
M. Cuturi. Sinkhorn distances: Lightspeed computation of optimal
transport. In Advances in Neural Information Processing Systems,
pages 2292–2300, 2013. 5
W. Förstner and B. Moonen. A Metric for Covariance Matrices, pages
299–309. Springer Berlin Heidelberg, Berlin, Heidelberg, 2003. 4
J. Friedman, T. Hastie, and R. Tibshirani. Sparse inverse covariance
estimation with the graphical lasso. Biostatistics, 9(3):432–441, 2008.
2
K. Friston, B. Sengupta, and G. Auletta. Cognitive dynamics: From
attractors to active inference. Proceedings of the IEEE, 102(4):427–
445, 2014. 1
Y. Gong, L. Wang, R. Guo, and S. Lazebnik. Multi-scale orderless
pooling of deep convolutional activation features. In Computer Vision
- ECCV 2014 - 13th European Conference, Zurich, Switzerland,
September 6-12, 2014, Proceedings, Part VII, pages 392–407, 2014.
2
K. Grauman and T. Darrell. Fast contour matching using approximate
earth mover’s distance. In Proceedings of the IEEE Computer Society
Conference on CVPR, volume 1, 2004. 5
J. R. Hershey and P. A. Olsen. Approximating the Kullback Leibler
divergence between Gaussian mixture models. In IEEE International
Conference on Acoustics, Speech and Signal Processing, volume 4,
2007. 3
W. James and J. Stein. Estimation with quadratic loss. In J. Neyman,
editor, Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, pages 361–379, 1961. 2
H. Ling and K. Okada. An efficient earth mover’s distance algorithm
for robust histogram comparison. IEEE transactions on Pattern
Analysis and Machine Intelligence, 29(5):840–853, 2007. 5
G. Macindoe. Hybrid algorithms for efficient Cholesky decomposition
and matrix inverse using multicore CPUs with GPU accelerators.
PhD thesis, University College London, 2013. 5
A. Mahendran and A. Vedaldi. Understanding deep image representations by inverting them. In Proceedings of the IEEE Computer
Society Conference on CVPR, 2015. 5
M. Moakher. A differential geometric approach to the geometric
mean of symmetric positive-definite matrices. SIAM J. Matrix Anal.
Appl., 26:35–747, 2005. 4
K. P. Murphy. Machine learning: a probabilistic perspective. MIT
Press, Cambridge, MA, 2012. 2
F. Nielsen and K. Sun. Guaranteed bounds on the Kullback-Leibler
divergence of univariate mixtures. IEEE Signal Processing Letters,
23(11):1543–1546, Nov 2016. 3

[22] Y. Rubner, C. Tomasi, and L. J. Guibas. A metric for distributions
with applications to image databases. In Computer Vision, 1998. Sixth
International Conference on, pages 59–66. IEEE, 1998. 5
[23] B. Sengupta and Y. Qian.
Pillar networks++: Distributed
non-parametric deep and wide networks.
arXiv preprint
arXiv:1708.06250, 2017. 5
[24] B. Sengupta and Y. Qian. Pillar networks for action recognition.
arXiv preprint arXiv:1707.06923, 2017. 5
[25] B. Sengupta, E. Vazquez, V. Simaiaki, M. Sasdelli, Y. Qian, M. Peniak, L. Netherton, and G. Delfino. Large-scale image analysis using
docker sandboxing. CoRR, abs/1703.02898, 2017. 1, 2
[26] S. Sra. Positive definite matrices and the s-divergence, 2015. 5
[27] A. Takatsu. On Wasserstein geometry of the space of Gaussian
measures. arXiv preprint arXiv:0801.2250, 2008. 4
[28] H. Tamura, S. Mori, and T. Yamawak. Textural features corresponding to visual perception. IEEE Transactions on systems, Man and
Cybernetics, page 460 473, 1978. 2
[29] A. Van den Oord, S. Dieleman, and B. Schrauwen. Deep contentbased music recommendation. In Advances in Neural Information
Processing Systems, pages 2643–2651, 2013. 1
[30] C. Villani. Optimal Transport: Old and New. Springer-Verlag, 2009.
1, 3, 4
[31] J. Yosinski, J. Clune, Y. Bengio, and H. Lipson. How transferable
are features in deep neural networks? In Advances in NIPS 27, pages
3320–3328. 2014. 5

