Behavior-Based Quality Assurance in Crowdsourcing Markets

Authors

  • Michael Feldman University of Zurich
  • Abraham Bernstein University of Zurich

Keywords:

crowdsourcing, quality assurance

Abstract

Quality assurance in crowdsourcing markets has appeared to be an acute problem over the last years. We propose a quality control method inspired by Statistical Process Control (SPC), commonly used to control output quality in production processes and characterized by relying on time-series data. Behavioral traces of users may play a key role in evaluating the performance of work done on crowdsourcing platforms. Therefore, in our experiment we explore fifteen behavioral traces for their ability to recognize the drop in work quality. Preliminary results indicate that our method has a high potential for real-time detection and signaling a drop in work quality.

Downloads

Published

2014-09-05

How to Cite

Feldman, M., & Bernstein, A. (2014). Behavior-Based Quality Assurance in Crowdsourcing Markets. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 2(1). Retrieved from https://ojs.aaai.org/index.php/HCOMP/article/view/13171