Title: Quality Enhancement by Weighted Rank Aggregation of Crowd Opinion,
Abstract: Expertise of annotators has a major role in crowdsourcing based opinion
aggregation models. In such frameworks, accuracy and biasness of annotators are
occasionally taken as important features and based on them priority of the
annotators are assigned. But instead of relying on a single feature, multiple
features can be considered and separate rankings can be produced to judge the
annotators properly. Finally, the aggregation of those rankings with perfect
weightage can be done with an aim to produce better ground truth prediction.
Here, we propose a novel weighted rank aggregation method and its efficacy with
respect to other existing approaches is shown on artificial dataset. The
effectiveness of weighted rank aggregation to enhance quality prediction is
also shown by applying it on an Amazon Mechanical Turk (AMT) dataset.