How to take multiple factors into account when evaluating a Game with a Purpose? How is player behaviour or participation influenced by different incentives? How does player engagement impact their accuracy in solving tasks? In this paper, we present a detailed investigation of multiple factors affecting the evaluation of a GWAP and we show how they impact on the achieved results. We inform our study with the experimental assessment of a GWAP designed to solve a multinomial classification task.
@cite_13: Data generated as a side effect of game play also solves computational problems and trains AI algorithms.
@cite_7: We introduce a new interactive system: a game that is fun and can be used to create valuable output. When people play the game they help determine the contents of images by providing meaningful labels for them. If the game is played as much as popular online games, we estimate that most images on the Web can be labeled in a few months. Having proper labels associated with each image on the Web would allow for more accurate image search, improve the accessibility of sites (by providing descriptions of images to visually impaired individuals), and help users block inappropriate images. Our system makes a significant contribution because of its valuable output and because of the way it addresses the image-labeling problem. Rather than using computer vision techniques, which don't work well enough, we encourage people to do the work by taking advantage of their desire to be entertained.
@cite_17: 
The basic metrics to evaluate GWAPs @cite_13 @cite_17 @cite_7 are global indicators computed as means over the entire data; while effective in summarizing the behaviour of GWAP players, those are very simple measures that do not tell the entire story: an analysis of data distribution and temporal evolution is usually required to get a deeper understanding of a GWAP.