Crowdsourcing Accurate and Creative Word Problems and Hints

Authors

  • Yvonne Chen University of Washington
  • Travis Mandel University of Washington
  • Yun-En Liu Enlearn
  • Zoran Popović University of Washington

DOI:

https://doi.org/10.1609/hcomp.v4i1.13295

Abstract

The current state-of-the-art method for generating educational content, such as math word problems and hints, is manual authoring by domain experts. Unfortunately, this is costly, time consuming, and produces content that lacks diversity. Attempts to automatically address the time and diversity issues through natural language generation still do not produce content that is sufficiently creative and varied. Crowdsourcing is a viable alternative - there has been a great deal of research on leveraging human creativity to solve complex problems, such as user interface design. However, these systems typically decompose complex tasks into subtasks. Writing a single word problem or hint is a small enough problem that it is unclear how to further break it down, but also far more complex than typical microtasks like image labeling. Therefore, it is not obvious how to apply these worker improvement methods or which ones are most effective (if at all). We build upon successful task design factors in prior work and run a series of iterative studies, incrementally adding different worker-support elements. Our results show that successive task designs improved accuracy and creativity.

Downloads

Published

2016-09-21

How to Cite

Chen, Y., Mandel, T., Liu, Y.-E., & Popović, Z. (2016). Crowdsourcing Accurate and Creative Word Problems and Hints. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 4(1), 12-21. https://doi.org/10.1609/hcomp.v4i1.13295