Title: Optimization Based Methods for Partially Observed Chaotic Systems,
Abstract: In this paper we consider filtering and smoothing of partially observed
chaotic dynamical systems that are discretely observed, with an additive
Gaussian noise in the observation. These models are found in a wide variety of
real applications and include the Lorenz 96' model. In the context of a fixed
observation interval $T$, observation time step $h$ and Gaussian observation
variance $\sigma_Z^2$, we show under assumptions that the filter and smoother
are well approximated by a Gaussian with high probability when $h$ and
$\sigma^2_Z h$ are sufficiently small. Based on this result we show that the
Maximum-a-posteriori (MAP) estimators are asymptotically optimal in mean square
error as $\sigma^2_Z h$ tends to $0$. Given these results, we provide a batch
algorithm for the smoother and filter, based on Newton's method, to obtain the
MAP. In particular, we show that if the initial point is close enough to the
MAP, then Newton's method converges to it at a fast rate. We also provide a
method for computing such an initial point. These results contribute to the
theoretical understanding of widely used 4D-Var data assimilation method. Our
approach is illustrated numerically on the Lorenz 96' model with state vector
up to 1 million dimensions, with code running in the order of minutes. To our
knowledge the results in this paper are the first of their type for this class
of models.