Machine Learning, Deepest Learning: Statistical Data Assimilation
  Problems
  We formulate a strong equivalence between machine learning, artificial
intelligence methods and the formulation of statistical data assimilation as
used widely in physical and biological sciences. The correspondence is that
layer number in the artificial network setting is the analog of time in the
data assimilation setting. Within the discussion of this equivalence we show
that adding more layers (making the network deeper) is analogous to adding
temporal resolution in a data assimilation framework.
  How one can find a candidate for the global minimum of the cost functions in
the machine learning context using a method from data assimilation is
discussed. Calculations on simple models from each side of the equivalence are
reported.
  Also discussed is a framework in which the time or layer label is taken to be
continuous, providing a differential equation, the Euler-Lagrange equation,
which shows that the problem being solved is a two point boundary value problem
familiar in the discussion of variational methods. The use of continuous layers
is denoted "deepest learning". These problems respect a symplectic symmetry in
continuous time/layer phase space. Both Lagrangian versions and Hamiltonian
versions of these problems are presented. Their well-studied implementation in
a discrete time/layer, while respected the symplectic structure, is addressed.
The Hamiltonian version provides a direct rationale for back propagation as a
solution method for the canonical momentum.
