Title: Accelerated Optimization in the PDE Framework: Formulations for the Active Contour Case,
Abstract: Following the seminal work of Nesterov, accelerated optimization methods have
been used to powerfully boost the performance of first-order, gradient-based
parameter estimation in scenarios where second-order optimization strategies
are either inapplicable or impractical. Not only does accelerated gradient
descent converge considerably faster than traditional gradient descent, but it
also performs a more robust local search of the parameter space by initially
overshooting and then oscillating back as it settles into a final
configuration, thereby selecting only local minimizers with a basis of
attraction large enough to contain the initial overshoot. This behavior has
made accelerated and stochastic gradient search methods particularly popular
within the machine learning community. In their recent PNAS 2016 paper,
Wibisono, Wilson, and Jordan demonstrate how a broad class of accelerated
schemes can be cast in a variational framework formulated around the Bregman
divergence, leading to continuum limit ODE's. We show how their formulation may
be further extended to infinite dimension manifolds (starting here with the
geometric space of curves and surfaces) by substituting the Bregman divergence
with inner products on the tangent space and explicitly introducing a
distributed mass model which evolves in conjunction with the object of interest
during the optimization process. The co-evolving mass model, which is
introduced purely for the sake of endowing the optimization with helpful
dynamics, also links the resulting class of accelerated PDE based optimization
schemes to fluid dynamical formulations of optimal mass transport.