Compact and discriminative visual codebooks are preferred in many visual recognition tasks. In the literature, a number of works have taken the approach of hierarchically merging visual words of an initial large-sized codebook, but implemented this approach with different merging criteria. In this work, we propose a single probabilistic framework to unify these merging criteria, by identifying two key factors: the function used to model class-conditional distribution and the method used to estimate the distribution parameters. More importantly, by adopting new distribution functions and or parameter estimation methods, our framework can readily produce a spectrum of novel merging criteria. Three of them are specifically focused in this work. In the first criterion, we adopt the multinomial distribution with Bayesian method; In the second criterion, we integrate Gaussian distribution with maximum likelihood parameter estimation. In the third criterion, which shows the best merging performance, we propose a max-margin-based parameter estimation method and apply it with multinomial distribution. Extensive experimental study is conducted to systematically analyse the performance of the above three criteria and compare them with existing ones. As demonstrated, the best criterion obtained in our framework achieves the overall best merging performance among the comparable merging criteria developed in the literature.
@cite_15: In patch-based object recognition, using a compact visual codebook can boost computational efficiency and reduce memory cost. Nevertheless, compared with a large-sized codebook, it also risks the loss of discriminative power. Moreover, creating a compact visual codebook can be very time-consuming, especially when the number of initial visual words is large. In this paper, to minimize its loss of discriminative power, we propose an approach to build a compact visual codebook by maxi- mally preserving the separability of the object classes. Furthermore, a fast algorithm is designed to accomplish this task effortlessly, which can hierarchically merge 10,000 visual words down to 2 in ninety seconds. Experimental study shows that the compact visual codebook created in this way can achieve excellent classification performance even after a considerable reduction in size.
@cite_4: This paper presents a new algorithm for the automatic recognition of object classes from images (categorization). Compact and yet discriminative appearance-based object class models are automatically learned from a set of training images. The method is simple and extremely fast, making it suitable for many applications such as semantic image retrieval, Web search, and interactive image editing. It classifies a region according to the proportions of different visual words (clusters in feature space). The specific visual words and the typical proportions in each object are learned from a segmented training set. The main contribution of this paper is twofold: i) an optimally compact visual dictionary is learned by pair-wise merging of visual words from an initially large dictionary. The final visual words are described by GMMs. ii) A novel statistical measure of discrimination is proposed which is optimized by each merge operation. High classification accuracy is demonstrated for nine object classes on photographs of real objects viewed under general lighting conditions, poses and viewpoints. The set of test images used for validation comprise: i) photographs acquired by us, ii) images from the Web and iii) images from the recently released Pascal dataset. The proposed algorithm performs well on both texture-rich objects (e.g. grass, sky, trees) and structure-rich ones (e.g. cars, bikes, planes)
@cite_25: We present an approach to determine the category and location of objects in images. It performs very fast categorization of each pixel in an image, a brute-force approach made feasible by three key developments: First, our method reduces the size of a large generic dictionary (on the order of ten thousand words) to the low hundreds while increasing classification performance compared to k-means. This is achieved by creating a discriminative dictionary tailored to the task by following the information bottleneck principle. Second, we perform feature-based categorization efficiently on a dense grid by extending the concept of integral images to the computation of local histograms. Third, we compute SIFT descriptors densely in linear time. We compare our method to the state of the art and find that it excels in accuracy and simplicity, performing better while assuming less.
This section reviews the supervised compact codebook creation methods in @cite_25 @cite_15 @cite_4 , with the focus on @cite_4 which inspires our work. As shown in @cite_15 , compact codebook creation can essentially be casted as a large-scale discrete optimization problem, subject to a criterion related to the discriminative power of the resultant compact codebook. Due to the difficulty of efficient and global optimization, hierarchically merging visual words is often adopted in the literature. That is, two words are identified at each level of the hierarchy such that merging them will optimize a given criterion. Let @math denote a visual codebook consisting of @math words. Let @math be the resultant codebook after merging the @math th and @math th words. The corresponding histogram for the @math th sample is denoted by @math , and its @math th bin is @math , where @math . Also, @math is the class label of a training sample. In this paper, the criteria in @cite_25 @cite_15 @cite_4 are termed AIB, CSM and UVD in short, respectively.