Abstract Designing discriminative powerful texture features robust to realistic imaging conditions is a challenging computer vision problem with many applications, including material recognition and analysis of satellite or aerial imagery. In the past, most texture description approaches were based on dense orderless statistical distribution of local features. However, most recent approaches to texture recognition and remote sensing scene classification are based on Convolutional Neural Networks (CNNs). The de facto practice when learning these CNN models is to use RGB patches as input with training performed on large amounts of labeled data (ImageNet). In this paper, we show that Local Binary Patterns (LBP) encoded CNN models, codenamed TEX-Nets, trained using mapped coded images with explicit LBP based texture information provide complementary information to the standard RGB deep models. Additionally, two deep architectures, namely early and late fusion, are investigated to combine the texture and color information. To the best of our knowledge, we are the first to investigate Binary Patterns encoded CNNs and different deep network fusion architectures for texture recognition and remote sensing scene classification. We perform comprehensive experiments on four texture recognition datasets and four remote sensing scene classification benchmarks: UC-Merced with 21 scene categories, WHU-RS19 with 19 scene classes, RSSCN7 with 7 categories and the recently introduced large scale aerial image dataset (AID) with 30 aerial scene types. We demonstrate that TEX-Nets provide complementary information to standard RGB deep model of the same network architecture. Our late fusion TEX-Net architecture always improves the overall performance compared to the standard RGB network on both recognition problems. Furthermore, our final combination leads to consistent improvement over the state-of-the-art for remote sensing scene classification.
@cite_68: Synthetic aperture radar (SAR) image classification is a hot topic in the interpretation of SAR images. However, the absence of effective feature representation and the presence of speckle noise in SAR images make classification difficult to handle. In order to overcome these problems, a deep convolutional autoencoder (DCAE) is proposed to extract features and conduct classification automatically. The deep network is composed of eight layers: a convolutional layer to extract texture features, a scale transformation layer to aggregate neighbor information, four layers based on sparse autoencoders to optimize features and classify, and last two layers for postprocessing. Compared with hand-crafted features, the DCAE network provides an automatic method to learn discriminative features from the image. A series of filters is designed as convolutional units to comprise the gray-level cooccurrence matrix and Gabor features together. Scale transformation is conducted to reduce the influence of the noise, which integrates the correlated neighbor pixels. Sparse autoencoders seek better representation of features to match the classifier, since training labels are added to fine-tune the parameters of the networks. Morphological smoothing removes the isolated points of the classification map. The whole network is designed ingeniously, and each part has a contribution to the classification accuracy. The experiments of TerraSAR-X image demonstrate that the DCAE network can extract efficient features and perform better classification result compared with some related algorithms.
@cite_53: Deep learning methods such as convolutional neural networks (CNNs) can deliver highly accurate classification results when provided with large enough data sets and respective labels. However, using CNNs along with limited labeled data can be problematic, as this leads to extensive overfitting. In this letter, we propose a novel method by considering a pretrained CNN designed for tackling an entirely different classification problem, namely, the ImageNet challenge, and exploit it to extract an initial set of representations. The derived representations are then transferred into a supervised CNN classifier, along with their class labels, effectively training the system. Through this two-stage framework, we successfully deal with the limited-data problem in an end-to-end processing scheme. Comparative results over the UC Merced Land Use benchmark prove that our method significantly outperforms the previously best stated results, improving the overall accuracy from 83.1 up to 92.4 . Apart from statistical improvements, our method introduces a novel feature fusion algorithm that effectively tackles the large data dimensionality by using a simple and computationally efficient approach.
@cite_16: Detecting small objects such as vehicles in satellite images is a difficult problem. Many features (such as histogram of oriented gradient, local binary pattern, scale-invariant featuretransform, etc.) have been used to improve the performance of object detection, but mostly in simple environments such as those on roads. proposed that no satisfactory accuracy has been achieved in complex environments such as the City of San Francisco. Deep convolutional neural networks (DNNs) can learn rich features from the training data automatically and has achieved state-of-the-art performance in many image classification databases. Though the DNN has shown robustness to distortion, it only extracts features of the same scale, and hence is insufficient to tolerate large-scale variance of object. In this letter, we present a hybrid DNN (HDNN), by dividing the maps of the last convolutional layer and the max-pooling layer of DNN into multiple blocks of variable receptive field sizes or max-pooling field sizes, to enable the HDNN to extract variable-scale features. Comparative experimental results indicate that our proposed HDNN significantly outperforms the traditional DNN on vehicle detection.
A large majority of published works use DNNs trained on patches extracted from satellite images. DNNs are usually not trained on databases of full sized satellite images (1 to several GB per image) due to memory limitations, even on powerful GPU servers. CNNs are the most commonly used deep learning architectures for the classification of optical @cite_16 and SAR @cite_68 satellite images. Because large datasets of satellite images with high quality labels are not easily available, most of the earlier works utilized pre-trained DNNs that were trained on computer vision benchmark datasets (ImageNet), not on satellite images @cite_53 .