Experiments on Motivational Feedback for Crowdsourced Workers


  • Tak Yeon Lee University of Maryland
  • Casey Dugan IBM T.J. Watson Research, Cambridge
  • Werner Geyer IBM T.J. Watson Research, Cambridge
  • Tristan Ratchford IBM T.J. Watson Research, Cambridge
  • Jamie Rasmussen IBM T.J. Watson Research, Cambridge
  • N. Sadat Shami IBM, Armonk
  • Stela Lupushor IBM, Armonk




Online Participation, Crowdsourcing, Motivation


This paper examines the relationship between motivational design and its longitudinal effects on crowdsourcing systems. In the context of a company internal web site that crowdsources the identification of Twitter accounts owned by company employees, we designed and investigated the effects of various motivational features including individual / social achievements and gamification. Our 6-month experiment with 437 users allowed us to compare the features in terms of both quantity and quality of the work produced by participants over time. While we found that gamification can increase workers’ motivation overall, the combination of motivational features also matters. Specifically, gamified social achievement is the best performing design over a longer period of time. Mixing individual and social achievements turns out to be less effective and can even encourage users to game the system.




How to Cite

Lee, T. Y., Dugan, C., Geyer, W., Ratchford, T., Rasmussen, J., Shami, N. S., & Lupushor, S. (2021). Experiments on Motivational Feedback for Crowdsourced Workers. Proceedings of the International AAAI Conference on Web and Social Media, 7(1), 341-350. https://doi.org/10.1609/icwsm.v7i1.14428