We then ask a separate set of Turkers to rate the stories for overall quality and the three improvement areas. All ratings are on a five-point scale. We collect two ratings per story, and throw out ratings that disagree by more than 2 points. A total of 11% of ratings were thrown out, leaving four metrics across 241 stories for analysis. 
 Question: How do they evaluate generated stories?
separate set of Turkers to rate the stories for overall quality and the three improvement areas