Why Stop Now? Predicting Worker Engagement in Online Crowdsourcing

Authors

  • Andrew Mao Harvard University
  • Ece Kamar Microsoft Research
  • Eric Horvitz Microsoft Research

DOI:

https://doi.org/10.1609/hcomp.v1i1.13076

Keywords:

engagement, attention, effort, volunteers, citizen science, machine learning, prediction, modeling

Abstract

We present studies of the attention and time, or engagement, invested by crowd workers on tasks. Consideration of worker engagement is especially important in volunteer settings such as online citizen science. Using data from Galaxy Zoo, a prominent citizen science project, we design and construct statistical models that provide predictions about the forthcoming engagement of volunteers. We characterize the accuracy of predictions with respect to different sets of features that describe user behavior and study the sensitivity of predictions to variations in the amount of data and retraining. We design our model for guiding system actions in real-time settings, and discuss the prospect for harnessing predictive models of engagement to enhance user attention and effort on volunteer tasks.

Downloads

Published

2013-11-03

How to Cite

Mao, A., Kamar, E., & Horvitz, E. (2013). Why Stop Now? Predicting Worker Engagement in Online Crowdsourcing. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 1(1), 103-111. https://doi.org/10.1609/hcomp.v1i1.13076

Issue

Vol. 1 (2013): First AAAI Conference on Human Computation and Crowdsourcing

Section

Full Papers