Title: Deep Mean Functions for Meta-Learning in Gaussian Processes,
Abstract: Fitting machine learning models in the low-data limit is challenging. The
main challenge is to obtain suitable prior knowledge and encode it into the
model, for instance in the form of a Gaussian process prior. Recent advances in
meta-learning offer powerful methods for extracting such prior knowledge from
data acquired in related tasks. When it comes to meta-learning in Gaussian
process models, approaches in this setting have mostly focused on learning the
kernel function of the prior, but not on learning its mean function. In this
work, we propose to parameterize the mean function of a Gaussian process with a
deep neural network and train it with a meta-learning procedure. We present
analytical and empirical evidence that mean function learning can be superior
to kernel learning alone, particularly if data is scarce.