Towards Data Quality Assessment in Online Advertising
  In online advertising, our aim is to match the advertisers with the most
relevant users to optimize the campaign performance. In the pursuit of
achieving this goal, multiple data sources provided by the advertisers or
third-party data providers are utilized to choose the set of users according to
the advertisers' targeting criteria. In this paper, we present a framework that
can be applied to assess the quality of such data sources in large scale. This
framework efficiently evaluates the similarity of a specific data source
categorization to that of the ground truth, especially for those cases when the
ground truth is accessible only in aggregate, and the user-level information is
anonymized or unavailable due to privacy reasons. We propose multiple
methodologies within this framework, present some preliminary assessment
results, and evaluate how the methodologies compare to each other. We also
present two use cases where we can utilize the data quality assessment results:
the first use case is targeting specific user categories, and the second one is
forecasting the desirable audiences we can reach for an online advertising
campaign with pre-set targeting criteria.
