Flexible Reward Plans to Elicit Truthful Predictions in Crowdsourcing

Abstract

We develop a flexible reward plan to elicit truthful predictive probability distribution over a set of uncertain events from workers.  In our reward plan, the principal can assign rewards for incorrect predictions according to her similarity between events.  In the spherical proper scoring rule, a worker's expected utility is represented as the inner product of her truthful predictive probability and her declared probability. We generalize the inner product by introducing a reward matrix that defines a reward for each prediction-outcome pair. We show that if the reward matrix is symmetric and positive definite, the spherical proper scoring rule guarantees the maximization of a worker's expected utility when she truthfully declares her prediction.