Pretraining. We collect three years of online news articles from June 2016 to June 2019. We filter out articles overlapping with the evaluation data on media domain and time range. We then conduct several data cleaning strategies. 
 Question: What did they pretrain the model on?
hree years of online news articles from June 2016 to June 2019