Gaussian Processes with Skewed Laplace Spectral Mixture Kernels for
  Long-term Forecasting
  Long-term forecasting involves predicting a horizon that is far ahead of the
last observation. It is a problem of high practical relevance, for instance for
companies in order to decide upon expensive long-term investments. Despite the
recent progress and success of Gaussian processes (GPs) based on spectral
mixture kernels, long-term forecasting remains a challenging problem for these
kernels because they decay exponentially at large horizons. This is mainly due
to their use of a mixture of Gaussians to model spectral densities.
Characteristics of the signal important for long-term forecasting can be
unravelled by investigating the distribution of the Fourier coefficients of
(the training part of) the signal, which is non-smooth, heavy-tailed, sparse,
and skewed. The heavy tail and skewness characteristics of such distributions
in the spectral domain allow to capture long-range covariance of the signal in
the time domain. Motivated by these observations, we propose to model spectral
densities using a skewed Laplace spectral mixture (SLSM) due to the skewness of
its peaks, sparsity, non-smoothness, and heavy tail characteristics. By
applying the inverse Fourier Transform to this spectral density we obtain a new
GP kernel for long-term forecasting. In addition, we adapt the lottery ticket
method, originally developed to prune weights of a neural network, to GPs in
order to automatically select the number of kernel components. Results of
extensive experiments, including a multivariate time series, show the
beneficial effect of the proposed SLSM kernel for long-term extrapolation and
robustness to the choice of the number of mixture components.
