Tuning the Diversity of Open-Ended Responses From the Crowd

Authors

  • Walter Lasecki University of Rochester
  • Christopher Homan Rochester Institute of Technology
  • Jeffrey Bigham Carnegie Mellon University

DOI:

https://doi.org/10.1609/hcomp.v2i1.13199

Keywords:

Microtasks, Game Theory

Abstract

Crowdsourcing can solve problems beyond the reach of state-of-the-art fully automated systems. A common pattern found in many such systems is for the workers to discover, in parallel, a number of candidate solutions and then vote on the best one to pass forward, often within a fixed amount of time. We present the propose-vote-abstain mechanism for eliciting from crowd workers the proper balance between solution discovery and selection. Each crowd worker is given a choice among proposing an answer, voting among the answers proposed so far, or abstaining, i.e., doing nothing. When a stopping condition is reached, the mechanism returns the answer with the most votes. Workers are paid a base amount, with bonuses if they propose or vote for the winning answer.

Downloads

Published

2014-09-05

How to Cite

Lasecki, W., Homan, C., & Bigham, J. (2014). Tuning the Diversity of Open-Ended Responses From the Crowd. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 2(1), 36-37. https://doi.org/10.1609/hcomp.v2i1.13199