Title: Learning across scales - A multiscale method for Convolution Neural Networks,
Abstract: In this work we establish the relation between optimal control and training
deep Convolution Neural Networks (CNNs). We show that the forward propagation
in CNNs can be interpreted as a time-dependent nonlinear differential equation
and learning as controlling the parameters of the differential equation such
that the network approximates the data-label relation for given training data.
Using this continuous interpretation we derive two new methods to scale CNNs
with respect to two different dimensions. The first class of multiscale methods
connects low-resolution and high-resolution data through prolongation and
restriction of CNN parameters. We demonstrate that this enables classifying
high-resolution images using CNNs trained with low-resolution images and vice
versa and warm-starting the learning process. The second class of multiscale
methods connects shallow and deep networks and leads to new training strategies
that gradually increase the depths of the CNN while re-using parameters for
initializations.