Targeted change detection in remote sensing
images

arXiv:1803.05482v1 [cs.CV] 14 Mar 2018

Vladimir Ignatiev1 , Alexey Trekin1 , Viktor Lobachev1 , Georgy Potapov1 , and
Evgeny Burnaev1
Skolkovo institute of Science and technology, Moscow, Nobel st. 1, Russia,
aeronetlab@skoltech.ru,
WWW home page: http://crei.skoltech.ru/cdise/aeronet-lab/

Abstract. Recent developments in the remote sensing systems and image processing made it possible to propose a new method of the object
classification and detection of the specific changes in the series of satellite Earth images (so called targeted change detection). In this paper
we propose a formal problem statement that allows to use effectively
the deep learning approach to analyze time-dependent series of remote
sensing images. We also introduce a new framework for the development
of deep learning models for targeted change detection and demonstrate
some cases of business applications it can be used for.
Key words: remote sensing, change detection, satellite imagery, computer vision, deep learning

1 Problem overview
Earth observation systems have been developing during the last decades in terms
of quality of the imagery and frequency of the imaging [27]. However, until now
the daily basis imagery wasnt available for any location or area of interest.
Each satellites and UAV systems have their own limitations - viz. number of
satellites, weather and daylight for optical systems, vegetation for SAR systems,
not to say the commercial ones. That lead the remote sensing industry to the
situation where the amount of the data and the proclaimed frequency of the
repeating imaging could have given us full daily coverage of the earth surface by
the high-resolution imagery [23], but in the reality it turns out to be composed
of the diverse, inconsistent and often incomplete data. And yet that business
and technology trend raise the issues how all this daily imagery can be targeted
to the specific needs and used in a various types of applications. The technical
issues that can be raised following these business needs are:
– data fusion of different types of remote sensing data coming from different
sensors (optical, SAR, different resolution etc.) - to provide the maximum
capacity of operational monitoring in any time or weather conditions [18, 19]
– detection of changes in the time series of imagery and in a very short time
spans to provide near real time valuable information related to the objects

2

Vladimir Ignatiev et al.

of interest. As the traditional approaches fail to make this massive dataflow
a reliable source of the useful answers we should propose the new machine
learning approaches that can handle it and provide the automated analysis
that can compete to the human one by accuracy
The business needs lead to the concept of the targeted change detection
(TCD) that is consolidated detection and classification of the changes in order
to obtain only the interesting for the problem (target) changes rather than all
the changes in general [25, 26].
Our motivation in this paper is to build a general description of change
detection (CD) problem taking into account characteristics of input data - series
of remote sensing (RS) images - and information about “targets” obtained from
classification of specific business applications of the respective territories such as
“transportation & logistic”, “residential areas”, “building & construction” etc.
Having formalized problem statement it is possible to develop overall framework
that will reduce costs and simplify stages of collecting data, model construction
and integration in workflow of business solutions.
1.1 Business applications of CD
Commercial services producing analytics based on series of multitemporal remote sensing images begin occupying significant share in such market areas as
smart farming [2, 3, 4], monitoring of security zones of linear objects that have
enormous transport network [9, 10, 7], monitoring of protected areas [5, 6, 8], assessment of damage caused by natural and man-made disasters [?, ?], analytical
solutions for consulting agencies in geo-marketing [12, 13].
Our survey of the commercial solutions based on analysis of RS images reveals
the following problems:
–
–
–
–

only a number of applications have algorithmic solutions;
most of solutions do not have enough accuracy;
the plenty of problems do not have any reliable solutions;
companies are in search of new frameworks that will anticipate their risks or
will make precise and timely estimation of losses.

It is obvious that each of use cases has its own demands and requirements
to the changes that should be detected. For example, if one is going to monitor
security zones of pipelines, he should find out the list of targets, changes of
which will be detected. Each target means a specific type of objects that has
the characteristic linear size and this value defines the spatial resolution of RS
images to be used. Most of real business applications requires to process with RS
images of high and very high spatial resolution from 5 cm/pixel to 10 m/pixel.
1.2 RS datasets available for CD algorithms
The idea of making general framework providing development of robust algorithms for detecting changes for the specific applied problem requires building

Targeted change detection

3

representative datasets. There are scarce datasets for change detection on RS images nowadays [14, 15, 16, 17]. Only few of existing datasets can be implemented
within the framework of targeted change detection [14, 17]. Most of them have
not enough coverage or time spread to learn and test algorithms using modern
techniques of machine learning and deep learning. So the one of the challenges
of our work is to collect and build a number of datasets dedicated to the task of
targeted CD.
1.3 Current methods and algorithms
Solutions proposed in the latest papers could be divided into several groups
according to the used models and algorithms, to market domains and to RS
images in use. Traditional methods of change detection by series of satellite
images are typically limited in several aspects:
– using just two images for justification with ground truth;
– do not take into account usability of changes in specific use case;
– only empirical approaches for selection of the appropriate RS images for real
applications;
– no automated end-to-end solutions for business cases.
One of the first applied research in this area was conducted by W.Roper and
S.Dutta [29]. They made an attempt to understand which kind of sensor system
aerial or satellite is better in a sense of probability of detection unauthorized
intrusion onto a pipeline right-of-way. The main conclusion of this work is that
one should use aerial imagery when the imaging frequency is lower than once per
month, but starting from once per day imaging frequency - probability of change
detection with satellite imagery is higher. There were used standard algorithms
of classification built-in such software packages as ENVI [28].
The future works were focused on improving change detection algorithms
themselves, but the idea of targeted CD were pushed into the background. So,
the best classical approaches are methods based on anomaly detection framework on time series of multispectral low-resolution satellite images and so called
spectral indices [34], methods based on Markov Random Fields and global optimization on graphs [35, 36, 37], approaches using object-based segmentation
with post-classification of changes [38, 39, 40] and methods based on Multivariate Alteration Detection [41, 30]. All the mentioned approaches shows quiet good
results in quality, but either use multi framed series of low resolution satellite
images or use high resolution images, but the same two pairs of images for both
training and testing.
Latest research achievements in computer vision made it possible to use deep
neural networks for accurate semantic segmentation objects on daily images. The
next step was to transfer this technology on remote sensing data. Actually, its
ongoing research. This results are the basement for development new algorithms
of CD. It can be emphasised the most successful studies in this direction. A.M.
Amin et al. [42] proposed a method for binary classification of changes based on
CNN with input two high resolution satellite images (0.65 - 2.62 m/pixel) gives

4

Vladimir Ignatiev et al.

the accuracy by Kappa coefficient - 0.876. The second approach by Y. Chu et al.
[43], based on clustering analysis of difference of two images (0.65 - 2.62 m/pixel)
uses deep belief network model with restricted Boltzmann machine architecture
shows the accuracy by Kappa coefficient from 0.905 to 0.987. The main advantage of this method that it allows to avoid radiometric correction of images, they
outperform the standard approaches and could be adopted for targeted change
detection in different use cases. Deep learning techniques need a lot of computational resources for training process and could be unstable for analysis of new
images, but the last problem may be eliminated having a representative dataset
for model training.
Further researches described in [30, 37, 45, 32, 31] give promising results, but
most of them share the same drawback: low amount of data used for the method
validation. The of creation a large and reliable dataset is one of the keys to the
development of robust and reliable algorithms for the targeted change detection.
One of the important issues is that many real problems do not need any
changes to be detected, but only the interesting, problem-specific ones. This
implies a challenging task to develop a method that could extract only the
specified classes of changes. The deep neural networks appear to be the most
promising approach to the problem, but the current solutions are highly limited
by the amount of the data available for the training of the models.

2 Remote sensing data for change detection
In order to understand the capability of the modern space imagery we need to
examine its characteristics. As stated in the previous section, according to the
target sizes we need the images in high (1-10 meters) and very high (less than
1 meter) spatial resolution. A typical high resolution satellite in 2018 has 4 or
more spectral bands and a panchromatic image of higher spatial resolution (see
table 2). A completely new approach to the data acquisition is demonstrated
by the PlanetScope satellite group, that consists of a large group of very small
satellites (3-unit CubeSat), that can acquire far more imagery than traditional
satellites of comparable mass and cost, however at the cost of less data quality:
they have less spectral bands and the noise level is higher [20].

Visible channels
Near infrared channels
SW infrared channels
Multispectral resolution, m
Panchromatic resolution, m
Daily coverage, millions sq.km/ day

SPOT-6 PlanetScope Worldview-3
4
3
6
1
0/1
2
0
0
8
6.0
3.0 / 3.7
1.24 / 3.72
1.5
0.31
3
150
0.68

Table 1. Overview of the technical specifications of the modern RS satellites

Targeted change detection

5

Let us call a set of images taken by one space or aerial instrument at one flight
and covering the same territory a “multi-layer image”. We need this term do
differentiate from the multispectral image where each layer represents a spectral
band and the panchromatic layer is not included into the multispectral image. In
our case we add it as well as any other information represented in a raster form
as a layer to our image representation, as we want to collect all the information
within a single instance of a multi-layer image.
I = I1 , ..., IC is a multi-layer image with C bands (channels). Every band Ii
is a single 2-dimensional matrix with the size (Xi , Yi ). Different bands can have
different spatial resolution so, under the condition that all the bands must cover
the same territory, the sizes of the channels are different, but proportional by an
integer ratio. If we order the bands by the decrease of the size, their dimensions
will be Xi /Xi + 1 = Yi /Yi+1 ∈ N
For the task of change detection the images are organized in time series:
S = I 1 , ...I T , where each image I t , t = 1...T covers the same area, but different
images can be obtained by different instruments and have different characteristics. We intentionally put this restriction on the area, because we need not to
describe all the data available, but to construct the datasets that can be used by
the data analysis methods within the proposed framework. If the given data are
not fully spatially consistent, we can crop the extents and, if needed, divide the
time series into several separate ones of different spatial and temporal extent.

3 Proposed framework
This section discusses details of proposed framework and shows the example
of methods for targeted change detection in classical situation when two input
satellite images are used for classification of one type of changes.
3.1 Problem statement
Given the series of images S = I 1 , ...I T where each image I t is a multi-layer
image taken at the moment of time t, a parameterized description of changes
occurred in the series can be introduced as C = F (S). The exact definition of
the operator F is done at the post-processing stage and is beyond the scope of
this paper. It is assumed that the calculation of the vector C will use only the
set of objects of pair-wise changes {rij }, so the problem goal can be reduced to
the search of rij - the change between the images I i and I j . The pair change
object definition includes:
– Category of detected changes;
– The set of polygonal domains (in physical coordinates) where the changes are
detected.
Evaluation of the detected change is done with one of the measures:
– Intersection over Union;

6

Vladimir Ignatiev et al.

– The pair Recall-Precision;
– F1-measure.
All three measures assume availability of the ground truth changes for the investigated pair of images. Each measure can be calculated either per unit area
or per object; the latter is specified heuristically.
3.2 Description of the framework for TCD applications
Our framework is restricted neither by the type of remote sensing data, nor methods, algorithms, but it does take into account information about the “targets”
of the applied area.
The question of determining the context of using the notion “change” is
removed by establishing a one-to-one correspondence of the model or algorithm
for detecting changes and the business case to be solved. Robustness of the
created algorithms is guaranteed by creating reference sets of markup of changes
for a series of different images and applying modern methods of in-depth training
on data generated from reference sets.
As one can see on the scheme depicted on the figure 1 there are three stages
in proposed framework: preparing dataset, training a model and analyzing series
of images for pair-wise targeted change detection. Dataset for specific business
application should be constructed in accordance with the formal mathematical
problem statement described above. It means that one should load series of
images concerning the size of targets, the characteristic time period of occurrence
of changes, type of images and so on. This data are combined with external markups of changes of objects (targets) under consumption. The model for pair-wise
targeted change detection is chosen from the set of neural networks architectures
prepared in advance and is trained and validated on the dataset built of the
first stage. The process of analyzing new series of images starts from building
map of changes for each pair of images inside the series. After that there is a
post-processing and verification procedures that resulted in recommendation,
forecasts or alerting reports with detected changes, that can be visualized in
interactive mode.
3.3 Method of TCD based on deep learning techniques
Our method is fundamentally different from all the methods listed in section
1.3 because it takes into account information about targets of the application
under consumption, is based on deep neural network trained and validated on
representative dataset, can be automatically adopted to a new territory.
The approach we are developing is based on a specific architecture of the
convolutional network which has had a great success in solving the semantic
segmentation problems. In the mathematical problem statement, a general formal description of the detected changes is introduced for a pair of images; the
description being applied to all datasets analyzed. The neural network architecture makes it possible to use the pre-trained parameters of the network and
adapt it to new datasets with an additional training at a very little expense.

Targeted change detection

Fig. 1. Scheme of the proposed framework

7

8

Vladimir Ignatiev et al.

The technique can be applied to all pairs of the images in the series; the
post-processing is then used to build a more common object that describes the
image evolution within the series. The post-processing can be specific for each
business application and does not depend on the deep learning technique used
at the early stages of analysis.

4 Conclusion
In this paper we have summarized the knowledge about the business applications
of change detection, known methods of the image processing and deep learning
and the possibilities and limitations of the RS data to propose a generalized
framework that can embed the solutions for different problems in the area of
the targeted change detection in the remote sensing data. The proposed framework to be implemented in our future research will facilitate the integration of
the current and future data sources and the advanced algorithms of the data
processing.

References
1. PlanetScope Flock, http://space.skyrocket.de/doc sdat/flock-1.htm
2. Gamaya, https://gamaya.com/
3. DigitalGlobe Services, https://www.digitalglobe.com/industries/agriculture
4. Exact Farming, www.exactfarming.com
5. Sovzond, https://sovzond.ru/
6. Unmanned Systems, http://unmanned.ru/en/
7. Orbital Eye - PIMSyS, https://orbitaleye.nl
8. Zala Aero Group, http://zala.aero/
9. I-Cubed, http://www.i3.com/
10. Satellite Imaging Corporation, www.satimagingcorp.com
11. European Space Agency, http://www.esa.int/ESA
12. OOO Rilos, https://geomatrix-retail.com/
13. Geomarketing & Location Intelligence, https://www.gim.be/en/productsservices/gim-expertise/geomarketing-location-intelligence
14. Bourdis, N., Marraud, D., & Sahbi, H. (2011, July). Constrained optical flow
for aerial image change detection. In Geoscience and Remote Sensing Symposium
(IGARSS), 2011 IEEE International (pp. 4176-4179). IEEE.
15. SAR Change Detection Dataset, http://fenggao.sinaapp.com/san dataset.html
16. Google Earth Engine, https://earthengine.google.com/datasets/
17. CDnet, http://www.changedetection.net/
18. Dong, J., Zhuang, D., Huang, Y., & Fu, J. (2009). Advances in multi-sensor data
fusion: Algorithms and applications. Sensors, 9(10), 7771-7784.
19. Zhang, J. (2010). Multi-source remote sensing data fusion: status and trends. International Journal of Image and Data Fusion, 1(1), 5-24.
20. Santilli, G., Vendittozzi, C., Cappelletti, C., Battistini, S., & Gessini, P. (2018).
CubeSat constellations for disaster management in remote areas. Acta Astronautica.

Targeted change detection

9

21. https://www.intelligence-airbusds.com/files/pmedia/public/r2928 9 int 012 spot67 en low-2.pdf
22. http://worldview3.digitalglobe.com/
23. https://www.planet.com/products/satelliteimagery/files/Planet Combined Imagery Product Specs December2017.pdf
24. https://qz.com/1126301/the-company-photographing-every-spot-of-land-onearth-every-single-day/
25. Fernndez-Prieto, D., & Marconcini, M. (2011). A novel partially supervised approach to targeted change detection. IEEE Transactions on Geoscience and Remote
Sensing, 49(12), 5016-5038.
26. Ye, S., Chen, D., & Yu, J. (2016). A targeted change-detection procedure by combining change vector analysis and post-classification approach. ISPRS Journal of
Photogrammetry and Remote Sensing, 114, 115-124.
27. Toth, C., & Jkw, G. (2016). Remote sensing platforms and sensors: A survey.
ISPRS Journal of Photogrammetry and Remote Sensing, 115, 22-36.
28. ENVI software, https://www.harris.com/solution/envi
29. Roper, W., & Dutta, S. (2005). Oil spill and pipeline condition assessment using
remote sensing and data visualization management systems. In: 2005 ESRI User
Conference Proceedings, San Diego, California, pp. 1-25.
30. Wang, B., Choi, J., Choi, S., Lee, S., Wu, P., & Gao, Y. (2017). Image Fusion-Based
Land Cover Change Detection Using Multi-Temporal High-Resolution Satellite Images. Remote Sensing, 9(8), 804.
31. Huang, S., Ramirez, C., Kennedy, K., Mallory, J., Wang, J., & Chu, C. (2017).
Updating land cover automatically based on change detection using satellite images:
case study of national forests in Southern California. GIScience & Remote Sensing,
54(4), 495-514.
32. Mcdermid, G. J., Pape, A., Chubey, M. S., & Franklin, S. E. (2003, October).
Object-oriented analysis for change detection. In Proceedings of the 25th Canadian
symposium on remote sensing, Montral, Canada (pp. 14-17).
33. Almutairi, A., & Warner, T. A. (2010). Change detection accuracy and image
properties: a study using simulated data. Remote Sensing, 2(6), 1508-1529.
34. Cai S., & Liu, D. (2015). Detecting Change Dates from Dense Satellite Time Series
Using a Sub-Annual Change Detection Algorithm. Remote Sensing, 7, 8705-8727.
doi:10.3390/rs70708705.
35. Vakalopoulou, M., Karantzalos, K., Komodakis, N., & Paragios, N. (2015). Simultaneous registration and change detection in multitemporal, very high resolution
remote sensing data. In Proceedings of the IEEE Conference on Computer Vision
and Pattern Recognition Workshops (pp. 61-69).
36. Yu, H., Yang, W., Hua, G., Ru, H., & Huang, P. (2017). Change Detection Using
High Resolution Remote Sensing Images Based on Active Learning and Markov
Random Fields. Remote Sensing, 9(12), 1233.
37. Gu, W., Lv, Z., & Hao, M. (2017). Change detection method for remote sensing
images based on an improved Markov random field. Multimedia Tools and Applications, 76(17), 17719-17734.
38. Jianya, G., Haigang, S., Guorui, M., & Qiming, Z. (2008). A review of multitemporal remote sensing data change detection algorithms. The International
Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences,
37(B7), 757-762.
39. Huang, L., Fang, Y., Zuo, X., & Yu, X. (2015). Automatic change detection method
of multitemporal remote sensing images based on 2D-Otsu algorithm improved by
firefly algorithm. Journal of Sensors, 2015.

10

Vladimir Ignatiev et al.

40. Vittek, M., Brink, A., Donnay, F., Simonetti, D., & Descle, B. Land cover change
monitoring using Landsat MSS/TM satellite image data over west africa between
1975 and 1990. Remote Sensing 2014; 6 (1): 658-676.
41. Jabari, S., & Zhang, Y. (2016, April). Building change detection using multi-sensor
and multi-view-angle imagery. In IOP Conference Series: Earth and Environmental
Science (Vol. 34, No. 1, p. 12-18). IOP Publishing.
42. El Amin, A. M., Liu, Q., & Wang, Y. (2016, July). Convolutional neural network
features based change detection in satellite images. In First International Workshop
on Pattern Recognition (Vol. 10011, p. 100110W). International Society for Optics
and Photonics.
43. Chu, Y., Cao, G., & Hayat, H. (2016). Change Detection of Remote Sensing Image
Based on Deep Neural Networks.
44. Lyu, H., Lu, H., & Mou, L. (2016). Learning a transferable change rule from a
recurrent neural network for land cover change detection. Remote Sensing, 8(6),
506.
45. Sakurada, K., Okatani, T., & Deguchi, K. (2013, June). Detecting changes in 3D
structure of a scene from multi-view images captured by a vehicle-mounted camera.
In Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on
(pp. 137-144). IEEE.

