Learning to Scale Payments in Crowdsourcing with PropeRBoost

Abstract

Motivating workers to provide significant effort has been recognized as an important issue in crowdsourcing. It is important not only to compensate worker effort, but also to discourage low-quality workers from participating. Several proper incentive schemes have been proposed for this purpose; they are either based on gold tasks or on peer consistency in individual tasks. As the rewards cannot become negative, these schemes have difficulty in achieving zero expected reward for random answers. We describe a novel boosting scheme, ProperRBoost, that improves the efficiency of existing incentive schemes by making a better separation between incentives for high and low quality work, and effectively discourages random answers by assigning them near minimal average rewards. We show the actual performance of the boosting scheme through simulations of various worker strategies.