Quality Control for Crowdsourced Enumeration Tasks

Authors

  • Shunsuke Kajimura The University of Tokyo
  • Yukino Baba National Institute of Informatics
  • Hiroshi Kajino The University of Tokyo
  • Hisashi Kashima Kyoto University

Abstract

Quality control is one of the central issues in crowdsourcing research. In this paper, we consider a quality control problem of crowdsourced enumeration tasks that request workers to enumerate possible answers as many as possible. Since workers neither necessarily provide correct answers nor provide exactly the same answers even if the answers indicate the same idea, we propose a two-stage quality control method consisting of the answer clustering stage and the reliability estimation stage.

Downloads

Published

2014-09-05

How to Cite

Kajimura, S., Baba, Y., Kajino, H., & Kashima, H. (2014). Quality Control for Crowdsourced Enumeration Tasks. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 2(1). Retrieved from https://ojs.aaai.org/index.php/HCOMP/article/view/13180