Is there an established methodology for human expert evaluation of machine learning predictions?
Measuring the inter-annotator agreement is one of the ways to measure the reliability of the annotations.

(I assume that there are also other methods but I'm not that knowledgeable about the topic)