Incentives to Counter Bias in Human Computation

Authors

  • Boi Faltings EPFL
  • Radu Jurca Google
  • Pearl Pu EPFL
  • Bao Duy Tran EPFL

Abstract

In online labor platforms such as Amazon Mechanical Turk, a good strategy to obtain quality answers is to take aggregate answers submitted by multiple workers, exploiting the wisdom of crowds. However, human computation issusceptible to systematic biases which cannot be corrected by using multiple workers.We investigate a game-theoretic bonus scheme, called peer truth serum (PTS), to overcome this problem. We report on the design and outcomes of a set of experiments to validate this scheme. Results show  peer truth serum can indeed correct the biases and increase the answer accuracy by up to 80%.

Downloads

Published

2014-09-05

How to Cite

Faltings, B., Jurca, R., Pu, P., & Tran, B. D. (2014). Incentives to Counter Bias in Human Computation. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 2(1). Retrieved from https://ojs.aaai.org/index.php/HCOMP/article/view/13145