Scene recognition is an image recognition problem aimed at predicting the category of the place at which the image is taken. In this paper, a new scene recognition method using the convolutional neural network (CNN) is proposed. The proposed method is based on the fusion of the object and the scene information in the given image and the CNN framework is named as FOS (fusion of object and scene) Net. In addition, a new loss named scene coherence loss (SCL) is developed to train the FOSNet and to improve the scene recognition performance. The proposed SCL is based on the unique traits of the scene that the 'sceneness' spreads and the scene class does not change all over the image. The proposed FOSNet was experimented with three most popular scene recognition datasets, and their state-of-the-art performance is obtained in two sets: 60.14 on Places 2 and 90.37 on MIT indoor 67. The second highest performance of 77.28 is obtained on SUN 397.
@cite_35: The Pascal Visual Object Classes (VOC) challenge is a benchmark in visual object category recognition and detection, providing the vision and machine learning communities with a standard dataset of images and annotation, and standard evaluation procedures. Organised annually from 2005 to present, the challenge and its associated dataset has become accepted as the benchmark for object detection. This paper describes the dataset and evaluation procedure. We review the state-of-the-art in evaluated methods for both classification and detection, analyse whether the methods are statistically different, what they are learning from the images (e.g. the object or its context), and what the methods find easy or confuse. The paper concludes with lessons learnt in the three year history of the challenge, and proposes directions for future improvement and extension.
@cite_18: Abstract In this paper, we present a feature description method called semantic descriptor with objectness (SDO) for scene recognition. Most existing scene representation methods exploit the characteristics of constituent objects in scenes with inter-class independence, which ignore the negative effects caused by the common objects among different scenes. The generic characteristics of the common objects cause some generality among different scenes, which weakens the discriminative characteristics among scenes. To address this problem, we exploit the correlations of object configurations among different scenes by the co-occurrence pattern of all objects across scenes to choose representative and discriminative objects which enhances the inter-class discriminability. Specifically, we capture the statistic information of objects appearing in each scene to compute the distribution of each object across scenes, which obtains the co-occurrence pattern of objects. Moreover, we represent the image descriptors with the occurrence probabilities of discriminative objects in image patches to eliminate the negative effects of common objects. To make image descriptors more discriminative, we discard the patches with non-discriminative objects to enhance the intra-class generalized characteristics. Experimental results on three widely used scene recognition datasets show that our method outperforms the state-of-the-art methods.
@cite_29: The ImageNet Large Scale Visual Recognition Challenge is a benchmark in object category classification and detection on hundreds of object categories and millions of images. The challenge has been run annually from 2010 to present, attracting participation from more than fifty institutions. This paper describes the creation of this benchmark dataset and the advances in object recognition that have been possible as a result. We discuss the challenges of collecting large-scale ground truth annotation, highlight key breakthroughs in categorical object recognition, provide a detailed analysis of the current state of the field of large-scale image classification and object detection, and compare the state-of-the-art computer vision accuracy with human accuracy. We conclude with lessons learned in the 5 years of the challenge, and propose future directions and improvements.
@cite_0: 
@cite_24: We present a new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of scene understanding. This is achieved by gathering images of complex everyday scenes containing common objects in their natural context. Objects are labeled using per-instance segmentations to aid in precise object localization. Our dataset contains photos of 91 objects types that would be easily recognizable by a 4 year old. With a total of 2.5 million labeled instances in 328k images, the creation of our dataset drew upon extensive crowd worker involvement via novel user interfaces for category detection, instance spotting and instance segmentation. We present a detailed statistical analysis of the dataset in comparison to PASCAL, ImageNet, and SUN. Finally, we provide baseline performance analysis for bounding box and segmentation detection results using a Deformable Parts Model.
@cite_34: Since scenes are composed in part of objects, accurate recognition of scenes requires knowledge about both scenes and objects. In this paper we address two related problems: 1) scale induced dataset bias in multi-scale convolutional neural network (CNN) architectures, and 2) how to combine effectively scene-centric and object-centric knowledge (i.e. Places and ImageNet) in CNNs. An earlier attempt, Hybrid-CNN[23], showed that incorporating ImageNet did not help much. Here we propose an alternative method taking the scale into account, resulting in significant recognition gains. By analyzing the response of ImageNet-CNNs and Places-CNNs at different scales we find that both operate in different scale ranges, so using the same network for all the scales induces dataset bias resulting in limited performance. Thus, adapting the feature extractor to each particular scale (i.e. scale-specific CNNs) is crucial to improve recognition, since the objects in the scenes have their specific range of scales. Experimental results show that the recognition accuracy highly depends on the scale, and that simple yet carefully chosen multi-scale combinations of ImageNet-CNNs and Places-CNNs, can push the stateof-the-art recognition accuracy in SUN397 up to 66.26 (and even 70.17 with deeper architectures, comparable to human performance).
Using object information in an image is the most utilized scene traits for scene recognition @cite_34 @cite_18 @cite_0 . When a particular object appears in an image, the chance of the image belonging to a certain category associated with the object increases. For example, if a TV is detected, the chance of the image being in a living room increases. In the previous works, the objects features were used for scene recognition instead of detecting the objects directly. To extract the object features, large image datasets for object recognition are used and they are ImageNet @cite_29 , PASCAL visual object classes (VOC) @cite_35 , or Microsoft COCO @cite_24 .