Convolutional Neural Networks and Data Augmentation for
Spectral-Spatial Classification of Hyperspectral Images
Jacopo Acquarelli1,2 , Elena Marchiori1 , Lutgarde M.C. Buydens2 , Thanh Tran3 , and
Twan van Laarhoven1
1 Radboud

University Nijmegen, Institute for Computing and Information Science
University Nijmegen, Institute for Molecules and Materials
3 Corbion, Gorinchem, The Netherlands

arXiv:1711.05512v1 [] 15 Nov 2017

2 Radboud

16th November 2017
Abstract
Spectral–spatial classification of remotely sensed hyperspectral images has been the subject of
many studies in recent years. Current methods achieve excellent performance on benchmark hyperspectral image labeling tasks when a sufficient number of labeled pixels is available. However, in the
presence of only very few labeled pixels, such classification becomes a challenging problem.
In this paper we propose to tackle this problem using convolutional neural networks (CNNs)
and data augmentation. Our newly developed method relies on the assumption of spectral-spatial
locality: nearby pixels in a hyperspectral image are related, in the sense that their spectra and their
labels are likely to be similar. We exploit this assumption to develop 1) a new data augmentation
procedure which adds new samples to the train set and 2) a tailored loss function which penalize
differences among weights of the network corresponding to nearby wavelengths of the spectra. We
train a simple single layer convolutional neural network with this loss function and augmented train
set and use it to classify all unlabeled pixels of the given image.
To assess the efficacy of our method, we used five publicly available hyperspectral images: Pavia
Center, Pavia University, KSC, Indian Pines and Salina. On these images our method significantly
outperforms other baselines. Notably, with just 1% of labeled pixels per class, on these dataset our
method achieves an accuracy of 99.5%, etc. Furthermore we show that our method improves over
other baselines also in a supervised setting, when no overlap between train and test pixels is allowed.
Overall our investigation demonstrates that spectral-spatial locality can be easily embedded in a
simple convolutional neural network through data augmentation and a tailored loss function1 .

1

Introduction

Hyperspectral images are useful for finding objects and identifying materials by taking advantage of
information enclosed in more spectral bands than the visible light. With the advance of technology,
sensors for making hyperspectral scans became cheaper and cheaper and their pixel resolution increased
too. Thus, hyperspectral images are more and more available and can be useful in different fields.
Territory control is one of these, where for instance, air pictures are shot from distance to determine the
distribution of different cultivations or urban areas or specific types of vegetation [1].
Image labeling is an important task for remotely sensed hyperspectral image exploitation [2]. The
goal is to assign a class label to each pixel, where a pixel is a whole spectrum of wavelengths. One can
distinguish two main approaches for hyperspectral image classification: pixel-wise and spectral–spatial.
In the pixel-wise approach, pixels are independent and only pixels with available labels are used to
perform classification. In the spectral–spatial approach the neighborhood information of labeled pixels is
also exploited, hence both labeled and unlabeled pixels are used for classification. In machine learning,
this latter setting is called transductive learning [3].
As expected, methods based on the spectral–spatial approach are more accurate, because they use an
intrinsic locality property of hyperspectral images, namely the fact that nearby pixels are spatially related
[2]. Spectral–spatial methods are based on diverse techniques, such as the utilizing Markov random field
1 Source

code of our method is publicly available at https://bitbucket.org/TeslaH2O/cnn_hyperspectral

1

for spatial inter-pixel dependency information in clustering [4–6], the construction of discriminatory
features [7–10], the modification and fusion of classifiers [11, 12], label propagation, active learning and
semi-supervised learning [13, 14], the use of external unlabeled data [15] and deep (convolutional) neural
networks [16–21].
Various methods achieve excellent performance on benchmark hyperspectral image classification tasks
when a sufficient number of manually labeled pixels is provided. However, manual labeling is an expensive
task. Therefore recent research has shifted towards the development of methods for the more challenging
problem of hyperspectral image classification when only a few manually labeled pixels are available.
We propose to tackle this problem using neural networks and data augmentation. As neural network,
we introduce a very simple convolutional neural network (CNN) (see Figure 1) with a customized loss
function to constrain the weights of the network model, which we are learnt by optimizing such function.
Specifically, we constrain weights corresponding to nearby wavelengths of the spectra to assume similar
values (see Sec. 4.1). As data augmentation, we enlarge the train set using two new strategies: smoothingand label-based data augmentation (see Secs. 4.2 and 4.3). These strategies allow us to exploit spectralspatial locality and to artificially increase the number of training samples (see Figure 2). The first
strategy is based on the assumption that nearby pixels have similar spectra, the second one that nearby
pixels are likely to belong to the same class. Results of our experiments show that the newly developed
method leads to state of the art performance, in particular when very few manually labeled pixels are
available for training.
It is important to stress that spectral–spatial methods are developed and tested in a transductive
learning setting [3], where labeled train pixels and unlabeled test pixels are used to infer the class of all
test pixels. This is a natural learning setting for hyperspectral image labeling, because often a whole
image consisting of both manually labeled and unlabeled pixels is available.
The performance of spectral–spatial methods is expected to decrease if supervised learning is used instead of transductive learning. The reason is that in supervised learning only the manually labeled pixels
are assumed to be available, so there is no overlap between train and test pixels. This learning setting
is mainly used by pixel-wise methods. Transfer and supervised learning in the context of hyperspectral
image classification have recently been compared in [22]. The authors proposed a controlled random
sampling strategy of labeled pixels to guarantee no overlap between train and test pixels. This strategy
can be used to test the performance of spectral–spatial methods also in a supervised learning setting.
We conduct experiments in both the transductive and supervised learning settings, the latter with a new
controlled random sampling strategy. Technically, having no overlap between train and test pixels does
not guarantee that the samples are i.i.d., so the resulting setting is not a properly supervised learning.
Nevertheless, it provides clear evidence of the advantage of transductive learning in this context.
In summary, our main contributions are:
• an easy to interpret and implement spectral–spatial approach based on CNNs;
• new data augmentation strategies for exploiting spectral-spatial locality;
• a new controlled random sampling strategy for assessing the performance of spectral-spatial methods in a supervised learning setting.

2

Related Work

There are other methods for hyperspectral image classification that use CNNs. They use different
architectures according to the desired representation of the input data (single pixel, patch of pixels, cube
of pixels, etc.). Moreover, some methods use preprocessing like PCA to either build a more compact
set of input features or to extract additional information (e.g. edge detection). In addition to this, the
majority of the approaches use a limited train set which is built by sampling a percentage or a maximum
number of pixels for each class, leaving the rest of pixels as test set; while in other approaches, data are
simply divided into train and test set.
Such methods differ from the one proposed in this paper mainly because they are more involved,
using more complex deep architectures and/or preprocessing procedures. These methods include [18], a
deep CNN with 2D input patches and R-PCA, [23], a deep Stacked Auto-Encoder with 2D input patches
and PCA, [16], a contextual deep CNN, [24], a multi-hypotheses prediction, [10], a Low-rank Gabor
filtering method, [17], a deep CNN with 1D pixel spectra, [21], a deep CNN with 1D pixel spectra, 2D
pixel patches or 3D pixel cubes, [19] a deep CNN with 1D pixel spectra, and [25], a deep CNN with

2

Figure 1: Architecture example of a single layer CNN applied to a hyperspectral image. The inputs
of the CNN are pixels on which 1D convolutions with kernels are applied in the convolutional layer.
Afterwards the resulting feature maps are stacked and fed to the last layer of the network which output
is the class prediction corresponding to the input pixels.

Figure 2: Train data augmentation. Gaussian noise is a standard augmentation technique used also in
the original CNN).
uniform smoothing kernel and 1D pixel spectra. Fortunately, the authors of the latter method shared
the source code with us, which we could then use in our comparative experimental analysis.

3

Convolutional Neural Network

We consider a CNN with just a single hidden layer. Unlike commonly used CNN architectures in deep
learning, we do not use pooling or fully-connected layers, and just use one convolutional layer (see Figure
Figure 1). We call this baseline CNN.
We use as loss function for training the standard L2 regularized cross-entropy error function, which,
for a train set (x, y) of I pixels, is:
I

1Õ
LossCNN (W) = −
I i=1
|
+ λ1 ·



yi log ŷi + (1 − yi ) log(1 − ŷi )
{z

Cross-entropy Error Loss



}

(1)

kWk 2
|{z}

L2 regularization

Here ŷi ≡ ψ2 (w2 · ψ1 (w1 · xi )) is the network’s output, ψ1 (·) and ψ2 (·) are the activation functions, xi
3

is the i-th pixel, W = [w1, w2 ] are the weights from the input to the single hidden layer (w1 ) , and from
the hidden layer to the output (w2 ), and yi is the target label.
The standard parameters of CNN are:
• learning rate (η);
• momentum (default value used in experiments: 0.7);
• number of convolutional kernels (#kernels);
• size of the convolutional kernels (N);
• stride for the convolution (s);
• L2 regularization constant (λ1 ).
To enhance the robustness of CNN to perturbed versions of the data, we use a standard data
augmentation technique: add random noise to the original data. This approach is quite simple and
does not rely on strong assumptions regarding data, especially when a Gaussian noise is used. In fact,
assuming that a certain signal can be affected by Gaussian noise is the natural way to model the noise in
absence of any knowledge. The new spectrum of a pixel is generated by adding random Gaussian noise
to the original spectrum as follows:
(2)
x new = xismt
j +β·
where x new is the new spectrum generated by perturbing xi j by adding Gaussian noise with zero mean,
 = N (0, 1) and unit variance and β is a constant term that we fixed to 0.01. This procedure is applied
to all the pixels in the train set.
In the next section we describe how to enhance this simple CNN by exploiting spectral-spatial locality.

4

Spectral-Spatial Locality

In spectral-spatial classification, also unlabeled pixels of the test set can be used to train a classifier.
This setting is used in all recent state of the art methods mentioned in the introduction: for instance [21]
employs input patches whose central pixel is in the train set, and [10] applies an efficient Gabor filter to
a L-size neighborhood of train pixels.
Here we exploit the spectral-spatial locality in three ways: (1) by constraining weights of the neural
network corresponding to nearby wavelengths to assume similar values (constraining nearby weights, see
Sec. 4.1), (2) by generating pixels with smoothed spectra from neighbors of labeled pixels (smoothingbased data augmentation, see Sec. 4.2), and (3) by propagating the label of a pixel to its neighbors and
adding them to the train set (label-based data augmentation, see Sec. 4.3).
Note that in steps (2) and (3) unlabeled test pixels are likely to be used.
We describe each step in the sequel.

4.1

Regularization

We add a term to the CNN loss function, which penalizes large differences between values of adjacent weights, as done in [26]. Such term helps the network keeping neighboring input variables (i.e.
wavelengths of the pixel spectra) correlated by penalizing large differences between near weights. This
enforces the weights of the neural network to reflect the spectral-spatial locality of the corresponding
pixels: we don’t expect large variations because the value of the spectrum at a certain wavelength is
dependent on the neighboring wavelength values.
The augmented loss function consists of the CNN regularized cross-entropy error term plus our
regularization term which constrains nearby weights to assume similar values:
I

1Õ
Loss(W) = −
I i=1
|
+ λ1 ·



yi log ŷi + (1 − yi ) log(1 − ŷi )
{z

Cross-entropy Error Loss

+λ2 ·

2

kWk
|{z}

L2 regularization

4



}

(3)
2

kw1 − Shi f t(w1 )k
|
{z
}

Constraints on nearby weights

5.0
5

4.5
4.0

Value

Value

4

3

3.5
3.0
2.5
2.0

2

1.5
1
0.4

0.82

1.24

1.66

1.0
0.4

2.08

Wavelengths (µm)

0.82

1.24

1.66

2.08

Wavelengths (µm)

Figure 3: Two original (left) and two smoothed (right) spectra of neighbor pixels of the Indian Pines
datasets. The original pixel has been smoothed applying a Gaussian kernel centered on it.
Here I is the number of train samples, ŷi ≡ ψ2 (w2 · ψ1 (w1 · xi )) is the network’s output, ψ1 (·) and ψ2 (·)
are the activation functions, xi is the i-th pixel, W = [w1, w2 ] are the weights from the input to the single
hidden layer (w1 ), and from the hidden layer to the output (w2 ), yi is the target label and Shi f t(·) is an
operation that shifts the elements of an array one position to the left.

4.2

Smoothing-based Data Augmentation

Spectra of nearby pixels are assumed to be related because they are part of an image containing semantically homogeneous components, such as urban or rural areas [2].
Recent state of the art methods exploit this property in different ways, such as the use patches to
train a (deep) neural network [16, 18, 23], the generation of discriminatory features using Gabor filters
[10, 24] or the use of additive Gaussian noise in addition to linear combination of training pixels [25].
Here we just use a Gaussian smoothing filter, because of its simplicity and invariance to rotation of
the image.
Formally, given an input hyperspectral image x ∈ R H xW x M (where W is the width and H is the
height of the image in pixels; M is the number of wavelengths representing each pixel), we generate the
smoothed image x smt as
Í Í
0 0 2
i0
j 0 xi jk exp(−k(i, j) − (i , j )k /2σ)
smt
xi jk = Í Í
.
(4)
0 0 2
i0
j 0 exp(−k(i, j) − (i , j )k /2σ)
In practice, we restrict the sums to a distance 3σ from (i, j), since pixels further away have a negligible
contribution. Figure 3 shows two pixel spectra of the same class before and after spatial smoothing is
applied.

4.3

Label-based Data Augmentation

Finally we exploit spectral-spatial locality with data augmentation performed at a semantic level, by
assuming that neighbor pixels are likely to have the same class. According to this assumption the label
of pixels in the train set can be propagated to its neighbors. The resulting labeled neighbor pixels
are inserted in the train set, which becomes larger at the cost of introducing label-noise. Indeed, this
data augmentation procedure is likely to insert in the train set new pixels with an incorrect label, and
even copies of the same pixel labeled in different ways. In this way the network is trained using a
train set extended with pixels having uncertainty on their label. In order to keep the probability that
our assumption is wrong as low as possible, we randomly sample only a subset of pixels in the Moore
neighborhood of each pixel (consisting of its 8 surrounding pixels). In the attempt of balancing the train
set, we use a different probability for a neighbor pixel to be selected that depends on the number of
pixels of each class. The probability of selecting a pixel i is determined as follows:
Pselected (i) = max(0, (9 ∗ min(C) − Ci )/(8 ∗ min(C)))

(5)

where C = [C1, . . . , Cc ] is a vector containing the number of pixels belonging to each class, c is the number
of classes and i represents the i-th class.
5

In summary, our label-based data augmentation procedure can be summarized as follows:
1. identify a neighborhood for each of the original training pixels;
2. select a subset of pixels in each neighborhood, with probability as in (Eq. (5));
3. propagate the label of each training pixel to its selected neighbor pixels;
4. insert the selected pixels with inferred labels into the train data.

5

CNN-RSL

Our resulting method for hyperspectral image labeling, called CNN-RSL, incorporates into CNN the
three spectral-spatial locality components above described: Regularization (R), Smoothing-based Data
Augmentation (S), and Label-based Data Augmentation (L). The resulting algorithm, called RSL-CNN,
is summarized in pseudo-code below.
Algorithm 1: CNN-RSL
Input : Train set (xtr ain , ytr ain ),
Test data xtest
Output: Predicted labels of test data ŷtest
smt , x smt ] = S(x
1: [xtr
tr ain , xtest ) (see Eq. (4))
test
augain
smt ) (see Sec. 4.3)
2: xtr ain = L(xtr
ain
f inal
aug
smt }
3: xtr ain = {xtr ain, xtr ain, xtr
ain
f inal
4: model = train CNN with loss Eq. (3) on (xtr ain , y f inal )
smt
5: ŷtest = apply the model to predict the labels ofxtest
6: return ŷtest
In Algorithm 1: at line 1 the Smoothing procedure (S) is applied, at line 2 the Label-basedaugmentation procedure (L), and at line 4 the Regularized (R) Loss (see equation (3)) is minimized
on the generated train data to learn the weights of our CNN-model. At line 5 the model is applied to
predict the labels of test pixels.

5.1

Training

To learn the weights of the network which optimizes our loss function we use a commonly used approach:
’Glorot’ procedure for initializing the weights [27] and Stochastic Gradient Descent (SGD) [28] rule for
updating them using the (augmented) train set. The parameters of the neural networks and the range
of values used in our experiments are:
• number of kernels of the convolutional layer: #kernels ∈ {4, 8, 16, 32}
• size of kernels of the convolutional layer: N ∈ [2, 91]
• stride for the convolution: s ∈ [1, 4]
• parameters in the regularization terms: λ1, λ2 = 10n where n ∈ [−4, 4]
• learning rate: lr = 10n where n ∈ [−4, −1]
The parameters of SVM-RBF are instead:
• the Gaussian exponent constant γ ∈ 10n where n ∈ [−4, 4]
• the regularization constant C ∈ 10n where n ∈ [−4, 4]
In order to tune these parameters we used the standard Random Grid Search Cross-Validation framework (RGS-CV) [29]. Resulting values of the parameters are given in Table 1.
We also use RGS-CV to select the value of σ, the parameter of our Gaussian smoothing procedure,
using values in the set {1, 1.67, 2.33, 3, 3.67, 4.33, 5}.
Figure 4 illustrates the convergence behavior of our loss function during training of CNN on one of
the datasets used in the experiments. To asses convergence we use early stopping. The training stops
when the validation error does not decrease for at least 100 epochs.
6

Parameters
#kernels
N
s
λ1
λ2
lr
σ

Datasets
Pavia Center

Pavia University

KSC

Indian Pines

Salinas

32
47
1
0.001
0.1
0.001
2.33

32
35
1
0.01
0.1
0.001
2.33

16
51
1
0.01
0.1
0.001
3.00

16
53
1
0.001
0.1
0.001
3.67

32
49
1
0.001
0.1
0.001
4.33

Table 1: Parameter values of CNN-RSL obtained using RGS-CV. CNN-RSL was trained using 1%
pixels of each class.

Figure 4: Convergence behavior of the CNN-RSL loss function (average over 10 folds of cross-validation
on the Salinas dataset). x-axis: epoch, y-axis: error.

6

Experimental Results

We assess comparatively the performance of our method, in particular the added value of its three
spectral-spatial locality components (R,S and L). To this end we consider the following algorithms:
• CNN: a simple single layer convolutional neural network (see Sec. 3);
• CNN-RSL and its three variants CNN-SL, CNN-RS and CNN-RL obtained by removing spectralspatial locality component R, S and L, respectively;
• SVM-RBF: a support vector machine with radial basis function (RBF) kernel and its three variants SVM-RBF-S, SVM-RBF-L and SVM-RBF-SL obtained by adding S, L and both data
augmentation components, respectively.
• HL-ELM: a deep convolutional neural network for hyperspectral image labeling or which we were
able to retrieve the source code and its three variants HL-ELM-S, HL-ELM-L and HL-ELM-SL
obtained by adding S, L and both data augmentation components, respectively.
We then have 13 algorithms to test on 5 publicly available hyperspectral images2 :
• Pavia Center : obtained with a Reflective Optics System Imaging Spectrometer (ROSIS) sensor
during a flight campaign over Pavia, northern Italy (see Sec. 6)
• Pavia University: scanned using the ROSIS sensor during a flight campaign over Pavia, northern
Italy (see Sec. 6)
2 See

http://www.ehu.eus/ccwintco/index.php?title=Hyperspectral_Remote_Sensing_Scenes

7

(a) Pavia Center test
pixels

(b) Pavia Center train
pixels

(c) Pavia Center legend

(g) Salinas test
pixels

(h)
Salinas
train pixels

(i) Salinas legend

Image
Pavia Center
Pavia University
KSC
Indian Pines
Salinas

(d) Pavia University test pixels

(e) Pavia University train pixels

Size

# Features

# Classes

1096 × 715
610 × 340
512 × 614
145 × 145
512 × 217

102
103
176
220
224

9
9
13
8
16

(f) Pavia University legend

Table 2: Description of the hyperspectral images.
• Kennedy Space Center (KSC): obtained with the Airborne/Visible Infrared Imaging Spectrometer
(AVIRIS) sensor over the Kennedy Space Center, Florida (USA) (see Figure 6)
• Indian Pines: scanned using AVIRIS sensor over the Indian Pines test site in North-western Indiana
(USA) (see Figure 7)
• Salinas: scanned using AVIRIS sensor over Salinas Valley, California (USA) (see Sec. 6)
Characteristics of the images (size, number of features and number of classes) are given in Table Table 2.

6.1

Transductive Learning Setting

We test CNN-RSL in the transductive learning setting [3]: given an hyperspectral image with some
labeled pixels, infer the labels of all the other pixels. In this setting, unlabeled test pixels can also be
used as train data.
Existing methods using this setting such as [16, 17, 19], have been shown to achieve very good results
on these datasets when 200 labeled pixels for each class are provided. Also our method achieves excellent
performance in this setting, which improves significantly over the considered baselines, according to a
binomial test for comparing classifiers [30] (see Table Table 10).
Since labeling pixels is an expensive operation to perform, a more realistic and challenging experimental protocol, used e.g. in [10], involves the use of few labeled pixels, namely 1%, 2%, . . . , 5% labeled
pixels per class.
Results of experiments in this setting are given in Tables 3 to 7. Also in this setting CNN-RSL
achieves the best performance, with significant improvement over the other baselines.

8

Methods

% Train
1

CNN
97.90 ± 0.16*
CNN-RSL
99.50 ± 0.02
CNN-RS
99.30 ± 0.06*
CNN-SL
98.54 ± 0.97*
CNN-RL
98.52 ± 0.08*
SVM-RBF
84.37 ± 2.88*
SVM-RBF-S
99.18 ± 0.46*
SVM-RBF-L
91.41 ± 8.76*
SVM-RBF-SL 98.92 ± 0.49*
HL-ELM
96.24 ± 0.05*
HL-ELM-S
98.74 ± 0.15*
HL-ELM-L
96.24 ± 0.05*
HL-ELM-SL
99.08 ± 0.13*

2

3

4

5

98.39 ± 0.09*
99.71 ± 0.07
99.43 ± 0.15*
98.45 ± 1.28*
98.89 ± 0.12*
85.38 ± 1.88*
99.48 ± 0.36*
92.63 ± 5.45*
98.27 ± 0.71*
97.34 ± 0.01*
98.97 ± 0.16*
97.33 ± 0.07*
99.31 ± 0.02*

98.61 ± 0.06*
99.80 ± 0.06
99.57 ± 0.06*
98.89 ± 1.60*
99.02 ± 0.05*
85.97 ± 1.70*
99.58 ± 0.34*
89.45 ± 0.98*
98.59 ± 0.13*
97.77 ± 0.05*
99.11 ± 0.11*
97.82 ± 0.11*
99.46 ± 0.02*

98.72 ± 0.04*
99.80 ± 0.04
99.55 ± 0.08*
99.36 ± 1.07*
99.08 ± 0.07*
87.48 ± 2.12*
99.63 ± 0.33*
92.79 ± 0.90*
98.88 ± 0.02*
97.98 ± 0.08*
99.23 ± 0.07*
98.06 ± 0.07*
99.47 ± 0.04*

98.80 ± 0.05*
99.82 ± 0.05
99.64 ± 0.06*
99.29 ± 0.62*
99.17 ± 0.09*
87.14 ± 1.85*
99.66 ± 0.32*
90.80 ± 0.86*
99.42 ± 0.50*
98.21 ± 0.09*
99.27 ± 0.08*
98.21 ± 0.03*
99.50 ± 0.05*

Table 3: Pavia Center average classification accuracy of test data over 10 runs using 1-5% of randomly
sampled labeled pixels per class as train set. The best accuracy for each train set is indicated in bold. A
‘*’ means that the best accuracy is significantly better than the accuracy achieved by the corresponding
method according to a binomial test for comparing classifiers [30] (p-value<0.05).
The increase in accuracy with respect to CNN, SVM-RBF and HL-ELM, even though dependent
from the original dataset, is higher when fewer training pixels are used, where our smoothing- and labelbased data augmentation are beneficial. Clearly these two components help also to boost the performance
of SVM-RBF and HL-ELM, which remains inferior to that of CNN-RSL.
As expected, by increasing the number of training pixels per class, the average test accuracy of all
methods increases. CNN-RSL achieves higher accuracy than a method DLRGF-LS recently published
in [10]. The KSC and Indian Pines and the same percentages of training pixels per class are used also in
this work, thus we can compare our results with those. However we were not able to get the source code
of DLRGF-LS and so we are not able to perform a statistical test to verify if the results of CNN-SL are
significantly better than DLRGF-LS.
Smoothing-based data augmentation introduces spatial locality into each pixel spectrum by averaging
it with its neighbor pixels. Since neighbor pixels are likely to belong to the same area, spatial smoothing
makes nearby spectra to look more alike and eases the network classification task. Smoothing based data
augmentation has the largest impact on the test accuracy improvements of all datasets.
Label-based data augmentation has a clear beneficial effect on the performance of all baselines, as can
be seen by the neat drop in accuracy when this step is omitted (see results of CNN-RS). This confirms
the strong spatial locality of the considered hyperspectral images and the efficacy of the devised data
augmentation procedures to exploit such locality. However the label-based data augmentation does not
always improve the performances. In fact the Pavia University and Salinas datasets they do not benefit
from the introduction of label augmentation. We are not sure about what causes this phenomenon, but
we can hypothesize that it is due to an inappropriate class re-balancing introduced by label-based data
augmentation using the formula mentioned in Sec. 4.3. However both CNN-RS and CNN-RSL achieve
accuracies which are not significantly different. Moreover, even if CNN-RSL is not the best method for
the Pavia University and Salinas datasets, its accuracy is still better than HL-ELM and SVM-RBF
based methods.
The customized regularization term for constraining the weights of the neural network in order to
exploit spectral-spatial locality helps to achieve a higher classification accuracy, as can be seen by the
reduced accuracy of CNN-SL. This confirms the added value of such term and shows that spectral-spatial
locality has been successfully exploited.
Let us analyze more in detail results when only 1% labeled pixels per class are used as train set. In
Tables 5 and 6 the per class test accuracy of the KSC and Indian Pines datasets is reported. For both
the datasets, the accuracy achieved by CNN-RSL for more than half of the classes is higher than the
other methods. CNN-RSL also has the highest number of classes with the best accuracy. While for the
Indian Pines dataset the highest per class accuracies belong to CNN-based methods mostly, for the KSC
dataset instead, other methods are more competitive with CNN-based methods. This can be explained

9

Methods

% Train
1

CNN
89.3 ± 0.84*
CNN-RSL
94.64 ± 1.29
CNN-RS
95.16 ± 0.18
CNN-SL
91.40 ± 3.71*
CNN-RL
88.85 ± 1.42*
SVM-RBF
75.74 ± 3.19*
SVM-RBF-S
89.01 ± 9.37*
SVM-RBF-L
57.34 ± 2.71*
SVM-RBF-SL 91.76 ± 3.32*
HL-ELM
75.84 ± 2.23*
HL-ELM-S
90.84 ± 0.61*
HL-ELM-L
75.84 ± 2.23*
HL-ELM-SL
92.85 ± 0.87*

2

3

4

5

91.57 ± 0.63*
94.62 ± 0.76
95.67 ± 1.37
91.48 ± 6.54*
88.59 ± 0.58*
73.98 ± 0.54*
91.97 ± 7.79*
79.51 ± 1.24*
94.19 ± 0.96*
78.97 ± 2.26*
92.99 ± 0.56*
81.23 ± 2.10*
94.78 ± 0.36

92.48 ± 0.23*
95.35 ± 0.66
96.89 ± 0.45
92.60 ± 1.77*
88.29 ± 0.40*
74.78 ± 1.67*
93.13 ± 7.52*
82.61 ± 6.99*
95.09 ± 0.66*
82.15 ± 2.36*
94.01 ± 0.32*
84.51 ± 1.83*
95.73 ± 0.22*

92.79 ± 0.33*
96.89 ± 0.64
97.08 ± 0.80
92.68 ± 1.22*
87.82 ± 0.56*
75.39 ± 2.53*
93.80 ± 7.47*
82.87 ± 0.82*
95.54 ± 3.31*
83.32 ± 2.30*
94.78 ± 0.15*
85.62 ± 1.65*
96.13 ± 0.16*

92.91 ± 0.55*
96.72 ± 1.12
96.25 ± 1.31
93.54 ± 1.30*
87.85 ± 1.16*
75.95 ± 3.32*
94.30 ± 7.37*
93.30 ± 0.43*
95.40 ± 0.07*
86.97 ± 2.10*
95.20 ± 0.12*
86.97 ± 1.43*
96.31 ± 0.06

Table 4: Pavia University average classification accuracy of test data over 10 runs using 1-5% of randomly
sampled labeled pixels per class as train set. The best accuracy for each train set is indicated in bold. A
‘*’ means that the best accuracy is significantly better than the accuracy achieved by the corresponding
method according to a binomial test for comparing classifiers [30] (p-value<0.05).
by observing that some of the KSC classes are completely correctly classified by several methods, thus
having the same accuracy as CNN-RSL.
Running time of CNN-RSL depends on the number of pixels and on the σ used for the spatial
smoothing. Moreover, the necessary time for spatial smoothing is determined by both these factors and
takes more time than predicting the class of pixels. This is a disadvantage of CNN-RSL with respect to
algorithms which use deep architectures and no spatial smoothing. However, spatial smoothing can be
highly parallelized given that each pixel is smoothed independently from the other pixels. Consequently,
its running time can be drastically reduced [31]. In Figure 5 we report running time of our method
obtained using a single CPU of our cluster machine having 2300 MHz of clock speed for each CPU. The
values refer to the time needed for predicting a single pixel and they also include the preprocessing time.

6.2

Supervised Learning Setting

The transductive setting is used in almost all works on spectral-spatial classification of hyperspectral
images. Transductive learning takes advantage of information enclosed into unlabeled test data to achieve
higher accuracy. In order to see the effect of this extra information coming from unlabeled test pixels,
we set up an experiment in which CNN-RSL does not take advantage of it. Thus, in this experiment
we made sure that no test sample took part in the smoothing of selected train pixels.
Specifically, we randomly sample a patch of train pixels for each class. The test set is made of pixels
which are not in the train set. We used a patch of 7 × 7 pixels as training data for each class to ensure
to have enough train pixels (at most 49). Each of them was smoothed without using any test pixel.
No label-based data augmentation is applied since this step would make use of test samples. Note that
constraining the weights of the network through our penalization term does not involve the use of test
pixels.
We have to point out that the way methods (including the one we propose) are tested on these
hyperspectral images is not the one used in classical supervised learning settings. In standard supervised
learning settings, methods are tested on i.i.d datasets. So in the case of image datasets, methods should
be trained on a set of images which is independent from test set images. However, as far as we know, all
the recent literature on classification of hyperspectral image pixels does not use the standard supervised
learning settings. Instead a dataset is generally represented by a single image. To train the methods, a
subset of pixel is chosen and then the rest of the pixels of the image constitute the test set.
Test accuracy achieved by the resulting algorithms in this different setting is reported in Table 11.
As we can see, the smoothing-based data augmentation helps to achieve a higher accuracy for all the
methods we used. Since we took a single 7x7 patch of pixels the performances of the methods are much
lower than those reported in Tables 3 to 7. Since we do not use label-based data augmentation, our

10

Methods

% Train
1

CNN
80.50 ± 2.11*
CNN-RSL
91.34 ± 1.92
CNN-RS
79.17 ± 1.31*
CNN-SL
87.22 ± 1.67*
CNN-RL
85.62 ± 1.58*
SVM-RBF
68.38 ± 1.85*
SVM-RBF-S
90.45 ± 7.16
SVM-RBF-L
81.37 ± 3.05*
SVM-RBF-SL 89.08 ± 4.47
HL-ELM
79.52 ± 0.56*
HL-ELM-S
81.58 ± 3.61*
HL-ELM-L
79.52 ± 0.56*
HL-ELM-SL
88.43 ± 2.70*

2

3

4

5

84.23 ± 1.20*
95.00 ± 0.33
86.35 ± 4.53*
93.55 ± 1.68*
89.15 ± 0.61*
75.97 ± 3.74*
90.96 ± 5.83*
87.57 ± 1.20*
90.79 ± 0.56*
81.82 ± 1.44*
87.58 ± 1.06*
83.26 ± 0.47*
91.42 ± 0.49*

85.93 ± 1.02*
96.74 ± 0.66
88.58 ± 2.24*
94.68 ± 2.37*
91.15 ± 1.18*
79.95 ± 3.24*
93.78 ± 4.30*
87.50 ± 2.60*
91.04 ± 6.16
82.64 ± 0.88*
90.77 ± 0.97*
83.51 ± 0.37*
93.52 ± 1.05*

88.03 ± 1.06*
98.68 ± 0.61
90.50 ± 1.62*
96.64 ± 2.24*
92.76 ± 1.23*
78.90 ± 5.80*
95.59 ± 3.69*
91.10 ± 1.83*
96.68 ± 0.85*
83.90 ± 1.36*
92.08 ± 0.75*
85.27 ± 0.49*
95.43 ± 0.60*

88.75 ± 1.75*
99.33 ± 0.46
91.47 ± 1.26*
97.26 ± 2.23*
94.22 ± 0.51*
80.89 ± 4.46*
96.80 ± 3.64*
92.56 ± 2.15*
97.23 ± 0.55*
86.55 ± 0.97*
93.35 ± 0.17*
86.55 ± 0.58*
96.89 ± 0.17*

Table 5: KSC average classification accuracy of test data over 10 runs using 1-5% of randomly sampled
labeled pixels per class as train set. The best accuracy for each train set is indicated in bold. A ‘*’ means
that the best accuracy is significantly better than the accuracy achieved by the corresponding method
according to a binomial test for comparing classifiers [30] (p-value<0.05).
reference method is CNN-RS. It outperforms other methods on the KSC, Pavia Center and University
while on the Indian Pines and Salinas HL-ELM-S is better than CNN-RS.
Another controlled random sampling method was introduced in [22]. That method considers connected component areas in the image, consisting of pixels with equal class. For each such area, pixels
are randomly sampled. Each selected pixel and its 8 neighbors form the train set. Pixels in the rest of
the image are only used at test time. See [22] (Algorithm 1) for a detailed description of this method.
Although this procedure is interesting for assessing the performance of spectral-spatial methods, it is
unrealistic since one would have to know the class composition of the whole image in order to perform
the step ‘Selects all unconnected partitions P in the class c’. Therefore we consider this procedure for
selecting training set pixel unpractical.

7

Conclusion

In this paper we have introduced a simple approach based on convolutional neural networks and data
augmentation for spectral–spatial classification of remotely sensed hyperspectral images. The proposed
approach provides a modular and effective strategy for boosting the performance of convolutional neural
networks with spectral-spatial information and provides a novel way to model and integrate such information in neural network classifiers.
The main characteristic of the proposed approach is its capability to exploit spectral-spatial information at both the data level (through data augmentation) and at the classifier level (through the constraining of the weights of the classifier). We propose two types of data augmentation: smoothing-based,
which constructs new pixels from the spectra of the neighbors of the labeled pixels, and label-based data
augmentation, which expands the train set with neighbors of the labeled pixels. At classifier level, we
modify the loss function by inserting a term to penalize the difference among networks weights corresponding to nearby wavelengths of the spectra. An advantage of this approach is its modularity, which
favors qualitative analysis of the contribution of the single steps as well as the embedding of data augmentation techniques in other types of neural networks. We incorporate these steps into a very simple
CNN, and analyze the beneficial effect of our data augmentation strategy also on other methods based
on neural networks.
Our data augmentation approach uses unlabeled test data. This machine learning setting is called
transductive learning and it is the natural setting for hyperspectral image classification. In the supervised
setting, where no overlap between train and test data is allowed, our data augmentation strategy can
not be used. In such a setting, only our classifier-level step can be used, since it does not involve the
use of test data. Accuracy of methods is expected to deteriorate in the latter case. We have illustrated

11

Methods

% Train
1

CNN
54.52 ± 2.21*
CNN-RSL
86.65 ± 2.06
CNN-RS
69.17 ± 1.31*
CNN-SL
68.21 ± 0.94*
CNN-RL
65.23 ± 1.05*
SVM-RBF
58.18 ± 0.39*
SVM-RBF-S
77.76 ± 4.82*
SVM-RBF-L
65.81 ± 2.83*
SVM-RBF-SL 85.36 ± 2.72*
HL-ELM
66.25 ± 0.11*
HL-ELM-S
73.04 ± 0.39*
HL-ELM-L
66.25 ± 0.11*
HL-ELM-SL
82.17 ± 0.84*

2

3

4

5

60.10 ± 1.80*
91.60 ± 0.55
86.35 ± 4.53*
78.34 ± 3.40*
71.76 ± 0.64*
60.40 ± 0.25*
83.01 ± 4.31*
69.39 ± 0.57*
90.06 ± 1.86*
71.80 ± 1.19*
82.64 ± 0.85*
73.00 ± 0.08*
88.75 ± 0.15*

64.47 ± 1.45*
93.46 ± 0.45
88.58 ± 2.24*
78.61 ± 5.07*
76.02 ± 0.80*
61.51 ± 0.13*
86.38 ± 4.90*
70.29 ± 1.21*
92.90 ± 0.94*
74.15 ± 0.34*
86.26 ± 0.43*
73.81 ± 0.07*
91.35 ± 0.63*

67.62 ± 0.78*
95.02 ± 0.50
90.50 ± 1.62*
80.74 ± 4.39*
78.06 ± 1.33*
63.73 ± 0.07*
88.98 ± 4.83*
77.03 ± 2.75*
93.18 ± 1.98*
76.54 ± 0.37*
88.72 ± 0.55*
76.90 ± 0.07*
93.28 ± 0.44*

70.37 ± 0.65*
95.91 ± 0.18
91.47 ± 1.26*
80.72 ± 5.26*
80.01 ± 1.33*
64.27 ± 0.04*
89.44 ± 4.17*
75.93 ± 2.19*
94.05 ± 0.16*
78.11 ± 0.42*
90.45 ± 0.28*
78.11 ± 0.08*
94.30 ± 0.56*

Table 6: Indian Pines average classification accuracy of test data over 10 runs using 1-5% of randomly
sampled labeled pixels per class as train set. The best accuracy for each train set is indicated in bold. A
‘*’ means that the best accuracy is significantly better than the accuracy achieved by the corresponding
method according to a binomial test for comparing classifiers [30] (p-value<0.05).
this phenomenon using a new and realistic random sampling strategy introduced in our paper. Results
of our experiments demonstrate the added value of our approach also in a supervised learning setting.
Results of experiments with several hyperspectral images demonstrate that our new method can yield
excellent accuracy results, which are comparable or better than those obtained by other state of the art
methods.
Our approach considers a single image. In future work we intend to adapt the approach to multiple
images. For instance, in a dynamic setting, where time-series spectral images are given in order to study
seasonal changes of vegetation species, we intend to develop multi-channel convolutional neural networks
with a tailored loss function to enforce smooth change in time.
Source code of our method is publicly available to guarantee full reproducibility and direct usage.

12

Methods

% Train
1

CNN
87.81 ± 0.51*
CNN-RSL
94.94 ± 0.38
CNN-RS
96.37 ± 0.16
CNN-SL
93.69 ± 2.28*
CNN-RL
87.39 ± 0.78*
SVM-RBF
72.57 ± 1.98*
SVM-RBF-S
94.26 ± 5.47*
SVM-RBF-L
85.83 ± 0.28*
SVM-RBF-SL 96.31 ± 0.54
HL-ELM
87.72 ± 0.47*
HL-ELM-S
92.59 ± 0.60*
HL-ELM-L
87.72 ± 0.47*
HL-ELM-SL
93.78 ± 0.80*

2

3

4

5

89.44 ± 0.33*
96.66 ± 0.37
96.51 ± 0.91
94.19 ± 1.31*
87.78 ± 1.70*
73.26 ± 2.53*
96.46 ± 3.29*
84.40 ± 4.05*
96.47 ± 0.15
89.13 ± 0.65*
94.51 ± 0.56*
89.78 ± 0.38*
95.65 ± 0.77*

90.34 ± 0.30*
97.60 ± 0.52
97.14 ± 0.31
95.33 ± 0.51*
86.52 ± 1.81*
73.73 ± 2.74*
97.03 ± 3.04*
84.92 ± 3.36*
97.64 ± 0.62
89.65 ± 0.34*
95.59 ± 0.47*
89.99 ± 0.25*
96.67 ± 0.10*

90.85 ± 0.30*
97.95 ± 0.46
97.18 ± 0.51
95.17 ± 0.45*
86.23 ± 2.07*
74.25 ± 3.01*
97.30 ± 2.88*
83.68 ± 2.05*
97.46 ± 1.03
91.22 ± 0.39*
96.06 ± 0.51*
91.22 ± 0.31*
97.27 ± 0.03

91.19 ± 0.29*
98.45 ± 0.59
97.44 ± 0.45
96.41 ± 0.87*
86.28 ± 1.78*
74.75 ± 3.25*
97.52 ± 2.68*
85.69 ± 1.26*
97.55 ± 0.69*
91.95 ± 0.28*
96.36 ± 0.58*
91.95 ± 0.35*
97.61 ± 0.08*

76.26
75.83
78.66
61.04
36.48
39.82
94.17
60.09
79.77
82.71
99.28
72.43
97.71

100
92.92
98.81
68.67
62.26
76.55
100
97.18
83.66
98.75
100
84.10
100

HL-ELM-S

82.34
68.75
77.47
48.59
72.33
57.96
98.06
68.54
84.24
87.22
96.14
60.56
98.36

93.63
80.00
73.52
69.08
39.62
41.15
100
76.06
94.94
87.47
93.24
64.39
100

HL-ELM-SL

100
77.08
94.47
89.56
99.37
55.75
100
88.73
83.66
81.45
100
84.10
100

HL-ELM

SVM-RBF-SL

SVM-RBF-L

94.82
37.92
77.08
61.68
8.81
11.5
62.14
63.38
73.15
42.61
53.86
59.56
97.06

SVM-RBF-S

86.74
76.25
79.84
61.04
79.25
53.54
97.09
87.09
80.16
91.48
99.03
81.29
99.89

HL-ELM-L

100
92.92
98.81
59.44
79.25
77.88
100
40.61
94.94
91.73
100
76.26
100

SVM-RBF

100
79.17
94.47
55.82
18.87
62.83
100
85.92
83.66
91.73
100
64.59
100

CNN-RL

100
77.50
86.96
57.83
100
84.07
100
99.06
83.66
100
100
83.50
100

93.49
75.00
84.19
55.02
15.09
27.43
90.29
77.23
85.41
72.18
93.72
74.04
99.89

CNN-SL

CNN-RS

CNN

Classes
Scrub
Willow swamp
Cp hammarock
CP/oak
Slash pipe
Oak/broadleaf
Hardwood swamp
Graminoid marsh
Spartina marsh
Cattail marsh
Salt marsh
Mud Flats
Water

CNN-RSL

Table 7: Salina average classification accuracy of test data over 10 runs using 1-5% of randomly sampled
labeled pixels per class as train set. The best accuracy for each train set is indicated in bold. A ‘*’ means
that the best accuracy is significantly better than the accuracy achieved by the corresponding method
according to a binomial test for comparing classifiers [30] (p-value<0.05).

82.36
68.75
77.47
48.59
72.33
57.96
98.06
68.54
84.24
87.22
96.14
60.56
98.47

91.51
77.50
96.05
57.43
90.57
78.32
100
82.63
84.82
95.74
100
67.20
99.89

84.43
85.04
62.82
92.90
97.10
98.52
78.69
66.58
89.27
95.54
97.04
95.03

HL-ELM-SL

65.11
69.67
63.25
76.99
92.66
99.15
49.33
35.69
34.24
93.07
92.01
53.14

HL-ELM-L

88.11
79.66
73.50
82.64
86.01
99.15
63.31
88.02
54.51
98.02
96.81
77.49

HL-ELM-S

47.63
22.66
5.56
58.79
83.66
83.72
39.5
74.24
5.28
92.08
95.37
11.78

HL-ELM

64..97
65.04
90.6
71.76
89.34
97.04
60.02
36.93
38.33
97.03
90.97
58.38

SVM-RBF-SL

SVM-RBF-L

66.45
69.55
70.94
96.23
99.17
96.41
95.22
32.72
65.59
65.59
100
93.19

SVM-RBF-S

CNN-SL

CNN-RS
79.48
67.48
11.97
39.12
84.35
100
60.40
82.59
9.20
80.69
93.93
54.19

SVM-RBF

90.18
92.45
70.94
94.77
93.77
100
89.93
83.90
64.74
99.50
93.29
96.86

CNN-RL

60.23
40.19
11.54
34.52
84.07
94.29
37.84
67.37
9.03
95.05
94.81
21.73

CNN-RSL

Classes
Corn-notill
Corn-mintill
Corn
Grass-pasture
Grass-trees
Grass-pasture-mowed
Hay-windrowed
Oats
Soybean-notill
Soybean-mintill
Soybean-clean
Woods

CNN

Table 8: Single run per class test accuracy achieved using 1% of train samples per class of the KSC
dataset. The best accuracy for each class is indicated in bold.

63.84
71.86
48.72
78.45
89.06
91.97
56.65
45.72
50.26
96.04
93.69
49.48

65.61
66.75
19.66
76.78
89.06
89.22
65.18
80.04
37.99
100
97.20
47.91

63.98
71.99
48.72
78.66
89.34
92.39
56.70
45.94
50.43
96.04
94.09
49.48

75.99
85.87
63.25
93.31
93.49
91.97
70.92
76.90
62.52
99.50
97.12
66.23

Table 9: Single run per class test accuracy achieved using 1% of train samples per class of the Indian
Pines dataset. The best accuracy for each class is indicated in bold.

13

Methods

Datasets
Pavia Center

CNN-RSL
99.60 ± 0.13
SVM-RBF
96.01 ± 3.06*
SVM-RBF-SL
98.41 ± 0.17*
Hu et.al. [17]
—
Lee & Kwon [16]
—
HL-ELM-SL[25] 98.04 ± 0.09*
MH-KELM[24]
—

Pavia University

KSC

Indian Pines

Salinas

97.99 ± 0.05
78.31 ± 11.13*
90.24 ± 0.22*
92.56
—
95.74 ± 0.49*
80.07 ± 0.01*

99.80 ± 0.22
92.49 ± 4.57*
98.88 ± 0.12*
—
—
93.08 ± 0.66*
—

97.94 ± 0.37
71.70 ± 7.69*
94.98 ± 4.75*
—
92.06
95.40 ± 0.54*
91.75 ± 0.29*

98.32 ± 0.63
87.01 ± 3.23*
97.09 ± 2.77*
92.60
—
97.84 ± 0.09*
—

Table 10: Average classification accuracy of test data over 10 runs using 200 randomly sampled labeled
pixels per class as train set. The best accuracy for each dataset is indicated in bold. With the exception
of methods from [16, 17]. A ‘*’ means that the best accuracy is significantly better than the accuracy
achieved by the corresponding method according to a binomial test for comparing classifiers [30] (pvalue<0.05).

Time (ms)/pixel

2

1.5

1

0.5

0

1

1.67

2.33

σ

3

3.67

4.33

Figure 5: Prediction time in milliseconds per pixel (ms/pixel) depending on the Gaussian window size
σ.

Methods

Datasets
Pavia Center

CNN
88.48 ± 2.61*
CNN-RS
92.15 ± 0.65
SVM-RBF
75.06 ± 8.83*
SVM-RBF-S 84.18 ± 5.69*
HL-ELM[25]
84.38 ± 2.03*
HL-ELM-S
88.30 ± 1.82*

Pavia University

KSC

Indian Pines

Salinas

48.98 ± 3.53*
60.59 ± 1.94
37.46 ± 9.02*
51.33 ± 6.61*
49.52 ± 4.63*
50.10 ± 4.12*

61.86 ± 5.00
64.86 ± 5.37
34.01 ± 7.43*
50.51 ± 6.45*
51.89 ± 5.13*
62.96 ± 1.63

28.57 ± 4.56
44.34 ± 5.46
30.58 ± 4.54
30.75 ± 6.61
42.52 ± 7.56
44.74 ± 7.31

62.72 ± 4.56
75.14 ± 3.18
49.71 ± 3.64*
54.12 ± 9.54
71.87 ± 5.07
75.89 ± 1.25

Table 11: Average classification accuracy of test data over 10 runs in which we made sure that pixels
employed during the smoothing-based data augmentation would not have been used as test pixels. We
randomly sample a patch of 7x7 pixels per class for building the train set. The best accuracy for
each dataset is indicated in bold. A ‘*’ means that the best accuracy is significantly better than the
accuracy achieved by the corresponding method according to a binomial test for comparing classifiers
[30] (p-value<0.05).

14

(a) Test pixels

(b) Train pixels

(c) CNN predictions

(d) CNN-RSL predictions

(e) CNN-RS predictions

(f) CNN-SL predictions

(g) CNN-RL predictions

(h) SVM-RBF predictions

(i) SVM-RBF-S predictions

(j) SVM-RBF-L predictions

(k) SVM-RBF-SL predictions

(l) HL-ELM predictions

(m) HL-ELM-S predictions

(n) HL-ELM-L predictions

(o) HL-ELM-SL predictions

(p) Legend

Figure 6: Classification results for the KSC dataset using only 1% of the labeled pixels available per
class for training.

15

(a) Test pixels

(b) Train pixels

(c) CNN predictions

(d) CNN-RSL predictions

(e) CNN-RS predictions

(f) CNN-SL predictions

(g) CNN-RL predictions

(h) SVM-RBF predictions

(i) SVM-RBF-S predictions

(j) SVM-RBF-L predictions

(k) SVM-RBF-SL predictions

(l) HL-ELM predictions

(m) HL-ELM-S predictions

(n) HL-ELM-L predictions

(o) HL-ELM-SL predictions

(p) Legend

Figure 7: Classification results for the Indian Pines dataset using only 1% of the labeled pixels available
per class for training.

16

Acknowledgements
Thanks to Jeroen Jansen for reading the manuscript and providing comments. Thanks to the authors
of [25] for providing the source code of their method. We would also like to thank the reviewers of a
previous version of this manuscript for their insightful comments.

References
[1] H. Latifi, F. Fassnacht, and B. Koch, “Forest structure modeling with combined airborne
hyperspectral and lidar data,” Remote Sensing of Environment, vol. 121, pp. 10 – 25, 2012.
[Online]. Available: //www.sciencedirect.com/science/article/pii/S0034425712000508
[2] G. Camps-Valls, D. Tuia, L. Bruzzone, and J. A. Benediktsson, “Advances in hyperspectral image classification: Earth monitoring with statistical learning methods,” IEEE Signal Processing
Magazine, vol. 31, no. 1, pp. 45–54, 2014.
[3] A. Gammerman, V. Vovk, and V. Vapnik, “Learning by transduction,” in Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence. Morgan Kaufmann Publishers Inc., 1998,
pp. 148–155.
[4] D. H. Hoekman, M. A. M. Vissers, and T. N. Tran, “Unsupervised full-polarimetric sar data segmentation as a tool for classification of agricultural areas,” IEEE Journal of Selected Topics in
Applied Earth Observations and Remote Sensing, vol. 4, no. 2, pp. 402–411, June 2011.
[5] T. N. Tran, R. Wehrens, and L. M. Buydens, “Clustering multispectral images:
a
tutorial,” Chemometrics and Intelligent Laboratory Systems, vol. 77, no. 1, pp. 3 –
17, 2005, fESTSCHRIFT HONOURING PROFESSOR D.L. MASSART. [Online]. Available:
http://www.sciencedirect.com/science/article/pii/S0169743904001984
[6] T. N. Tran, R. Wehrens, D. H. Hoekman, and L. M. C. Buydens, “Initialization of markov random
field clustering of large remote sensing images,” IEEE Transactions on Geoscience and Remote
Sensing, vol. 43, no. 8, pp. 1912–1919, Aug 2005.
[7] P. Ghamisi, J. A. Benediktsson, and J. R. Sveinsson, “Automatic spectral–spatial classification
framework based on attribute profiles and supervised feature extraction,” IEEE Transactions on
Geoscience and Remote Sensing, vol. 52, no. 9, pp. 5771–5782, 2014.
[8] N. Falco, J. A. Benediktsson, and L. Bruzzone, “Spectral and spatial classification of hyperspectral
images based on ica and reduced morphological attribute profiles,” IEEE Transactions on Geoscience
and Remote Sensing, vol. 53, no. 11, pp. 6223–6240, 2015.
[9] L. He, Y. Li, X. Li, and W. Wu, “Spectral–spatial classification of hyperspectral images via spatial
translation-invariant wavelet-based sparse representation,” IEEE Transactions on Geoscience and
Remote Sensing, vol. 53, no. 5, pp. 2696–2712, 2015.
[10] L. He, J. Li, A. Plaza, and Y. Li, “Discriminative low-rank gabor filtering for spectral–spatial
hyperspectral image classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 55,
no. 3, pp. 1381–1395, 2017.
[11] M. A. Veganzones, G. Tochon, M. Dalla-Mura, A. J. Plaza, and J. Chanussot, “Hyperspectral
image segmentation using a new spectral unmixing-based binary partition tree representation,”
IEEE Transactions on Image Processing, vol. 23, no. 8, pp. 3574–3589, 2014.
[12] T. Lu, S. Li, L. Fang, X. Jia, and J. A. Benediktsson, “From subpixel to superpixel: A novel fusion
framework for hyperspectral image classification,” IEEE Transactions on Geoscience and Remote
Sensing, 2017.
[13] B. Sun, X. Kang, S. Li, and J. A. Benediktsson, “Random-walker-based collaborative learning for
hyperspectral image classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 55,
no. 1, pp. 212–222, 2017.

17

[14] Z. Wang, B. Du, L. Zhang, L. Zhang, and X. Jia, “A novel semisupervised active-learning algorithm
for hyperspectral image classification,” IEEE Transactions on Geoscience and Remote Sensing,
vol. 55, no. 6, pp. 3071–3083, 2017.
[15] R. Kemker and C. Kanan, “Self-taught feature learning for hyperspectral image classification,” IEEE
Transactions on Geoscience and Remote Sensing, vol. 55, no. 5, pp. 2693–2705, 2017.
[16] H. Lee and H. Kwon, “Contextual Deep CNN Based Hyperspectral Classification,” arXiv, vol.
1604.03519, pp. 2–4, 2016.
[17] W. Hu, Y. Huang, L. Wei, F. Zhang, and H. Li, “Deep convolutional neural networks for hyperspectral image classification,” Journal of Sensors, vol. 2015, 2015.
[18] K. Makantasis, K. Karantzalos, A. Doulamis, and N. Doulamis, “Deep Supervised Learning for
Hyperspectral Data Classification through Convolutional Neural Networks,” IGARSS 2015. 2015
IEEE International Geoscience and Remote Sensing Symposium. Proceedings \, pp. 4959–4962,
2015.
[19] V. Slavkovikj, S. Verstockt, W. D. Neve, S. V. Hoecke, and R. V. D. Walle, “Hyperspectral Image
Classification with Convolutional Neural Networks,” pp. 1159–1162, 2015.
[20] H. Liang and Q. Li, “Hyperspectral imagery classification using sparse representations of convolutional neural network features,” Remote Sensing, vol. 8, no. 2, 2016.
[21] Y. Chen, H. Jiang, C. Li, X. Jia, and P. Ghamisi, “Deep feature extraction and classification of
hyperspectral images based on convolutional neural networks,” IEEE Transactions on Geoscience
and Remote Sensing, vol. 54, no. 10, pp. 6232–6251, Oct 2016.
[22] J. Liang, J. Zhou, Y. Qian, L. Wen, X. Bai, and Y. Gao, “On the sampling strategy for evaluation of
spectral-spatial methods in hyperspectral image classification,” IEEE Transactions on Geoscience
and Remote Sensing, vol. 55, no. 2, pp. 862–880, 2017.
[23] Y. Chen, Z. Lin, X. Zhao, S. Member, G. Wang, and Y. Gu, “Deep Learning-Based Classification
of Hyperspectral Data,” vol. 7, no. 6, pp. 1–14, 2014.
[24] X. Ma, H. Wang, and J. Geng, “Spectral-spatial classification of hyperspectral image based on deep
auto-encoder,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,
vol. 9, no. 9, pp. 4073–4085, Sept 2016.
[25] Q. Lv, X. Niu, Y. Dou, J. Xu, and Y. Lei, “Classification of hyperspectral remote sensing image using
hierarchical local-receptive-field-based extreme learning machine,” IEEE Geoscience and Remote
Sensing Letters, vol. 13, no. 3, pp. 434–438, March 2016.
[26] J. Acquarelli, T. van Laarhoven, J. Gerretzen, T. N. Tran, L. M. Buydens, and E. Marchiori,
“Convolutional neural networks for vibrational spectroscopic data analysis,” Analytica Chimica
Acta, vol. 954, pp. 22 – 31, 2017. [Online]. Available: http://www.sciencedirect.com/science/
article/pii/S0003267016314842
[27] X. Glorot and Y. Bengio, “Understanding the difficulty of training deep feedforward neural networks,” in JMLR W&CP: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS 2010), vol. 9, May 2010, pp. 249–256.
[28] L. Bottou, “Large-scale machine learning with stochastic gradient descent,” in Proceedings of
COMPSTAT’2010, Y. Lechevallier and G. Saporta, Eds. Physica-Verlag HD, 2010, pp. 177–186.
[Online]. Available: http://dx.doi.org/10.1007/978-3-7908-2604-3_16
[29] J. Bergstra and Y. Bengio, “Random search for hyper-parameter optimization,” The Journal
of Machine Learning Research, vol. 13, pp. 281–305, Feb. 2012. [Online]. Available:
http://dl.acm.org/citation.cfm?id=2188385.2188395
[30] S. L. Salzberg, “On comparing classifiers: Pitfalls to avoid and a recommended approach,”
Data Mining and Knowledge Discovery, vol. 1, no. 3, pp. 317–328, 1997. [Online]. Available:
http://dx.doi.org/10.1023/A:1009752403260
[31] B. Cope et al., “Implementation of 2d convolution on fpga, gpu and cpu.”

18

