Crowdsourcing Quality Control for Item Ordering Tasks
DOI:
https://doi.org/10.1609/hcomp.v1i1.13106Abstract
One of the biggest challenges in crowdsourcing is quality control which is to expect high quality results from crowd workers who are not necessarily very capable nor motivated.In this paper, we consider item ordering questions, where workersare asked to arrange multiple items in the correct order. We propose a probabilistic generative model of crowd answers by extending a distance-based order model to incorporate worker ability, and give an efficient estimation algorithm.
Downloads
Published
2013-11-03
How to Cite
Matsui, T., Baba, Y., Kamishima, T., & Kashima, H. (2013). Crowdsourcing Quality Control for Item Ordering Tasks. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 1(1), 52-53. https://doi.org/10.1609/hcomp.v1i1.13106
Issue
Section
Works in Progress