Title: Models for Predicting Community-Specific Interest in News Articles,
Abstract: In this work, we ask two questions: 1. Can we predict the type of community
interested in a news article using only features from the article content? and
2. How well do these models generalize over time? To answer these questions, we
compute well-studied content-based features on over 60K news articles from 4
communities on reddit.com. We train and test models over three different time
periods between 2015 and 2017 to demonstrate which features degrade in
performance the most due to concept drift. Our models can classify news
articles into communities with high accuracy, ranging from 0.81 ROC AUC to 1.0
ROC AUC. However, while we can predict the community-specific popularity of
news articles with high accuracy, practitioners should approach these models
carefully. Predictions are both community-pair dependent and feature group
dependent. Moreover, these feature groups generalize over time differently,
with some only degrading slightly over time, but others degrading greatly.
Therefore, we recommend that community-interest predictions are done in a
hierarchical structure, where multiple binary classifiers can be used to
separate community pairs, rather than a traditional multi-class model. Second,
these models should be retrained over time based on accuracy goals and the
availability of training data.