We focus on the data shared by BIBREF3, and empirically test to what extent changes in the evaluation design affect the outcome of the human evaluation. In this study, we address three aspects that we consider to be particularly relevant for human evaluation of MT, with a special focus on testing human–machine parity: the choice of raters, the use of linguistic context, and the construction of reference translations. We empirically test and discuss the impact of these factors on human evaluation of MT in Sections SECREF3–SECREF5.  
 Question: What empricial investigations do they reference?
empirically test to what extent changes in the evaluation design affect the outcome of the human evaluation