Dataset Quality Analysis ::: Inter-Annotator Agreement (IAA)
To estimate dataset consistency across different annotations, we measure F1 using our UA metric with 5 generators per predicate. Dataset Quality Analysis ::: Dataset Assessment and Comparison
We assess both our gold standard set and the recent Dense set against an integrated expert annotated sample of 100 predicates.  Dataset Quality Analysis ::: Agreement with PropBank Data
It is illuminating to observe the agreement between QA-SRL and PropBank (CoNLL-2009) annotations BIBREF7.  
 Question: How was quality measured?
Inter-annotator agreement, comparison against expert annotation, agreement with PropBank Data annotations.