Problems with using personal feedback to motivate participation in an online psychological experiment? I am planning on running an online psychological experiment where participants must learn about various simulated environments and then make inferences about them. Because completing the experiment would require a nontrivial amount of attention and time, I have been toying with the idea of giving personal feedback at the end of the experiment (i.e., using gamification).
I am considering giving participants feedback regarding how their responses compare to our computational model's predictions and/or to the average. The hope is that receiving a &quot;score&quot;, even when it's made clear that such a score is not evaluative or absolute, will get people to pay more attention and will make them more likely to complete the task. The feedback would only be offered following the experiment itself, and its precise nature would not be revealed until that time. Moreover, I wouldn't specify how our predicted scores are derived. Calculating the exact values by hand would be extremely time consuming, so even if the derivation is explained I don't see how anyone could realistically bias their responses with that knowledge.
Questions

Does giving personal feedback raise any ethical problems?
How might it affect response patterns?
Will giving personal feedback elicit higher completion rates and more attentive responding, without any significant complications?
Ethics of Feedback:

The APA's code of ethics (2010) as well as the Advisory Group on Conducting Research on the Internet (AGCRI) report (2004) summarize ethical issues related to conducting offline and online psychological research.  Feedback is normal in psychological experiments, and researchers are encouraged to debrief participants, before, immediately after, or as soon as the research conducted reasonably allows without risking an effect on results.

Research proposals should undergo an Institutional Review Board (IRB) review prior to initiation.  A consideration with respect to feedback is if it might reasonably be construed by participants as being on a positive-negative dimension (ie , the feedback could be considered negative), then you may want to reconsider providing it, whereas if the feedback is on a neutral dimension (an example would be a personality trait such as placement on an introvert-extrovert dimension), then there is unlikely to be any concern.

Effect of Motivation:

Of course, feedback provided after task completion cannot affect task performance.  Presumably however, the promise or expectation of feedback at the end of a task may have an effect.  Without knowing the nature of the research, the general findings on the effect of incentives on results are that they are negligible if any.  For example, Mason and Watts (2009) find that "Accuracy ... is not reliably different for
different wages," and similarly, a review of online studies conducted on Mechanical Turk (2011) reports that "most workers are not motivated primarily by the financial returns and genuinely care about the quality of their work," and result validity is quite high after some basic spam filtering is applied.

Nonetheless, I would recommend testing the effect of your chosen incentive if at all possible.  This can be done either by comparing results from unincentivized participants to those of incentivized participants, or through validity checks - for example in surveys common practice is to include multiple questions with similar meanings but different wordings to check that they all get similar responses as a way of confirming that answers are legitimate and not just trying to get through the survey as fast as possible.

Completion Rates:

The effect of incentives on task completion in general is very mixed.  Online studies do consistently see greater participation for higher paid experiments, but for non-monetary incentives the results are more complex.  The overjustification effect is an example where incentives may actually reduce task completion rates or performance quality, though this effect is not very robust in practice.      Houmanfar and Hayes (1998) report that "the results indicated no effects of either private feedback or public feedback on task completion," Conrad et al (2010) find that "feedback can improve their experience though not necessarily their completion rates."

The difference between a simple score at the end of the experiment, and more typical notions of gamification may be considerable however.  Gamification in general does have an overall positive impact.    Gamification typically involves a variety of interactive features that come into play throughout the task.  So if at all possible, I would recommend incorporating say, a progress indicator, badge system, storyline, or any other gamification feature that can reasonably be added.  Note that gamification features are not uniformly successful, so some experimentation may be necessary to determine what works best in your particular situation.

Non-contingent Incentives:

One final note: You didn't mention if the feedback would be contingent on completion of the experiment.  While the idea of providing incentives only on task completion may seem intuitively sound, there is substantial evidence that non-contingent incentives are generally more effective.  For example, if you promise to provide a score only on task completion, then fewer people might attempt the task than if you promise to provide a score regardless, and even if completion rates are much lower you might get higher participation overall.  Say only 10 people attempt the task for a contingent incentive, and have a 50% completion rate, then you get 5 results.  On the other hand if 100 people attempt the task for a non-contingent incentive, and have a much lower 10% completion rate, then you get 10 results - twice as many.  Worth considering I'd say.