Adaptive Representation Selection in Contextual Bandit
  We consider an extension of the contextual bandit setting, motivated by
several practical applications, where an unlabeled history of contexts can
become available for pre-training before the online decision-making begins. We
propose an approach for improving the performance of contextual bandit in such
setting, via adaptive, dynamic representation learning, which combines offline
pre-training on unlabeled history of contexts with online selection and
modification of embedding functions. Our experiments on a variety of datasets
and in different nonstationary environments demonstrate clear advantages of our
approach over the standard contextual bandit.
