Data-Driven Off-Policy Estimator Selection: An Application in User
  Marketing on An Online Content Delivery Service
  Off-policy evaluation (OPE) is the method that attempts to estimate the
performance of decision making policies using historical data generated by
different policies without conducting costly online A/B tests. Accurate OPE is
essential in domains such as healthcare, marketing or recommender systems to
avoid deploying poor performing policies, as such policies may hart human lives
or destroy the user experience. Thus, many OPE methods with theoretical
backgrounds have been proposed. One emerging challenge with this trend is that
a suitable estimator can be different for each application setting. It is often
unknown for practitioners which estimator to use for their specific
applications and purposes. To find out a suitable estimator among many
candidates, we use a data-driven estimator selection procedure for off-policy
policy performance estimators as a practical solution. As proof of concept, we
use our procedure to select the best estimator to evaluate coupon treatment
policies on a real-world online content delivery service. In the experiment, we
first observe that a suitable estimator might change with different definitions
of the outcome variable, and thus the accurate estimator selection is critical
in real-world applications of OPE. Then, we demonstrate that, by utilizing the
estimator selection procedure, we can easily find out suitable estimators for
each purpose.
