Title: Machine Learning as Statistical Data Assimilation,
Abstract: We identify a strong equivalence between neural network based machine
learning (ML) methods and the formulation of statistical data assimilation
(DA), known to be a problem in statistical physics. DA, as used widely in
physical and biological sciences, systematically transfers information in
observations to a model of the processes producing the observations. The
correspondence is that layer label in the ML setting is the analog of time in
the data assimilation setting. Utilizing aspects of this equivalence we discuss
how to establish the global minimum of the cost functions in the ML context,
using a variational annealing method from DA. This provides a design method for
optimal networks for ML applications and may serve as the basis for
understanding the success of "deep learning". Results from an ML example are
presented.
When the layer label is taken to be continuous, the Euler-Lagrange equation
for the ML optimization problem is an ordinary differential equation, and we
see that the problem being solved is a two point boundary value problem. The
use of continuous layers is denoted "deepest learning". The Hamiltonian version
provides a direct rationale for back propagation as a solution method for the
canonical momentum; however, it suggests other solution methods are to be
preferred.