TY - JOUR AU - Madge, Chris AU - Yu, Juntao AU - Chamberlain, Jon AU - Kruschwitz, Udo AU - Paun, Silviu AU - Poesio, Massimo PY - 2019/10/28 Y2 - 2024/03/28 TI - Progression in a Language Annotation Game with a Purpose JF - Proceedings of the AAAI Conference on Human Computation and Crowdsourcing JA - HCOMP VL - 7 IS - 1 SE - Technical Papers DO - 10.1609/hcomp.v7i1.5276 UR - https://ojs.aaai.org/index.php/HCOMP/article/view/5276 SP - 77-85 AB - <p>Within traditional games design, incorporating progressive difficulty is considered of fundamental importance. But despite the widespread intuition that progression could have clear benefits in Games-With-A-Purpose (GWAPs)–e.g., for training non-expert annotators to produce more complex judgements– progression is not in fact a prominent feature of GWAPs; and there is even less evidence on its effects. In this work we present an approach to progression in GWAPs that generalizes to different annotation tasks with minimal, if any, dependency on gold annotated data. Using this method we observe a statistically significant increase in accuracy over randomly showing items to annotators.</p> ER -