The bag-of-features (BoF) model for image classification has been thoroughly studied over the last decade. Different from the widely used BoF methods which model images with a pre-trained codebook, the alternative codebook-free image modeling method, which we call codebookless model (CLM), attracts little attention. In this paper, we present an effective CLM that represents an image with a single Gaussian for classification. By embedding Gaussian manifold into a vector space, we show that the simple incorporation of our CLM into a linear classifier achieves very competitive accuracy compared with state-of-the-art BoF methods (e.g., Fisher Vector). Since our CLM lies in a high-dimensional Riemannian manifold, we further propose a joint learning method of low-rank transformation with support vector machine (SVM) classifier on the Gaussian manifold, in order to reduce computational and storage cost. To study and alleviate the side effect of background clutter on our CLM, we also present a simple yet effective partial background removal method based on saliency detection. Experiments are extensively conducted on eight widely used databases to demonstrate the effectiveness and efficiency of our CLM method. HighlightsWe show that codebookless model (CLM) is a very competitive alternative to the mainstream BoF model.Two well-motivated parameters are introduced to further improve our CLM.A joint low-rank learning and SVM is proposed on Gaussian manifold.The comprehensive experiments demonstrated the promising performance of our CLM.
@cite_14: We investigate the properties of a metric between two distributions, the Earth Mover's Distance (EMD), for content-based image retrieval. The EMD is based on the minimal cost that must be paid to transform one distribution into the other, in a precise sense, and was first proposed for certain vision problems by Peleg, Werman, and Rom. For image retrieval, we combine this idea with a representation scheme for distributions that is based on vector quantization. This combination leads to an image comparison framework that often accounts for perceptual similarity better than other previously proposed methods. The EMD is based on a solution to the transportation problem from linear optimization, for which efficient algorithms are available, and also allows naturally for partial matching. It is more robust than histogram matching techniques, in that it can operate on variable-length representations of the distributions that avoid quantization and other binning problems typical of histograms. When used to compare distributions with the same overall mass, the EMD is a true metric. In this paper we focus on applications to color and texture, and we compare the retrieval performance of the EMD with that of other distances.
@cite_3: Tensors are nowadays a common source of geometric information. In this paper, we propose to endow the tensor space with an affine-invariant Riemannian metric. We demonstrate that it leads to strong theoretical properties: the cone of positive definite symmetric matrices is replaced by a regular and complete manifold without boundaries (null eigenvalues are at the infinity), the geodesic between two tensors and the mean of a set of tensors are uniquely defined, etc. We have previously shown that the Riemannian metric provides a powerful framework for generalizing statistics to manifolds. In this paper, we show that it is also possible to generalize to tensor fields many important geometric data processing algorithms such as interpolation, filtering, diffusion and restoration of missing data. For instance, most interpolation and Gaussian filtering schemes can be tackled efficiently through a weighted mean computation. Linear and anisotropic diffusion schemes can be adapted to our Riemannian framework, through partial differential evolution equations, provided that the metric of the tensor space is taken into account. For that purpose, we provide intrinsic numerical schemes to compute the gradient and Laplace-Beltrami operators. Finally, to enforce the fidelity to the data (either sparsely distributed tensors or complete tensors fields) we propose least-squares criteria based on our invariant Riemannian distance which are particularly simple and efficient to solve.
@cite_31: We describe a new region descriptor and apply it to two problems, object detection and texture classification. The covariance of d-features, e.g., the three-dimensional color vector, the norm of first and second derivatives of intensity with respect to x and y, etc., characterizes a region of interest. We describe a fast method for computation of covariances based on integral images. The idea presented here is more general than the image sums or histograms, which were already published before, and with a series of integral images the covariances are obtained by a few arithmetic operations. Covariance matrices do not lie on Euclidean space, therefore we use a distance metric involving generalized eigenvalues which also follows from the Lie group structure of positive definite matrices. Feature matching is a simple nearest neighbor search under the distance metric and performed extremely rapidly using the integral images. The performance of the covariance features is superior to other methods, as it is shown, and large rotations and illumination changes are also absorbed by the covariance matrix.
@cite_10: This paper introduces a feature descriptor called shape of Gaussian (SOG), which is based on a general feature descriptor design framework called shape of signal probability density function (SOSPDF). SOSPDF takes the shape of a signal's probability density function (pdf) as its feature. Under such a view, both histogram and region covariance often used in computer vision are SOSPDF features. Histogram describes SOSPDF by a discrete approximation way. Region covariance describes SOSPDF as an incomplete parameterized multivariate Gaussian distribution. Our proposed SOG descriptor is a full parameterized Gaussian, so it has all the advantages of region covariance and is more effective. Furthermore, we identify that SOGs form a Lie group. Based on Lie group theory, we propose a distance metric for SOG. We test SOG features in tracking problem. Experiments show better tracking results compared with region covariance. Moreover, experiment results indicate that SOG features attempt to harvest more useful information and are less sensitive against noise.
@cite_11: Modeling image similarity for browsing and searching in voluminous image databases is a challenging task of nearly all content-based image retrieval systems. One promising way of defining image similarity consists in applying distance-based similarity measures on compact image representations. Beyond feature histograms and feature signatures, more general feature representations are mixture models of which the Gaussian mixture model is the most prominent one. This feature representation can be compared by employing approximations of the Kullback-Leibler Divergence. Although several of those approximations have been successfully applied to model image similarity, their applicability to mixture models based on high-dimensional feature descriptors is questionable. In this paper, we thus introduce the Signature Quadratic Form Distance to measure the distance between two Gaussian mixture models of high-dimensional feature descriptors. We show the analytical computation of the proposed Gaussian Quadratic Form Distance and evaluate its retrieval performance by making use of different benchmark image databases.
The codebookless model for directly modeling the statistics of local features has been studied in past decades. Rubner al @cite_14 introduced signatures for image representation, and proposed the Earth Mover's Distance for image matching which is robust but has high computational cost. Tuzel al @cite_31 for the first time used covariance matrices for representing regular image regions, and employed Affine-Riemannian metric which suffers from high computational cost @cite_3 . Gaussian model as image descriptor has been used for visual tracking @cite_10 , in which Gaussian models are matched based on the Riemannian metric, involving expensive operations to solve generalized eigenvalue problem. Going beyond Gaussian, Gaussian mixture model (GMM) is more informative and is used in image retrieval @cite_11 . However, GMM suffers from some limitations, such as high computational cost of matching methods and lacking of general criteria for model selection.