Human Judgments
Following BIBREF11 , BIBREF12 and the vast amount of previous work on semantic similarity, we ask nine undergraduate subjects to assess the similarity of 360 pairs of relations from a subset of Wikidata BIBREF8 that are chosen to cover from high to low levels of similarity. In our experiment, subjects were asked to rate an integer similarity score from 0 (no similarity) to 4 (perfectly the same) for each pair.  
 Question: How do they gather human judgements for similarity between relations?
By assessing similarity of 360 pairs of relations from a subset of Wikidata using an integer similarity score from 0 to 4