HodgeRank With Information Maximization for Crowdsourced Pairwise Ranking Aggregation

Authors

  • Qianqian Xu Institute of Information Engineering, CAS, Beijing; BICMR and School of Mathematical Sciences, Peking University, Beijing
  • Jiechao Xiong BICMR and School of Mathematical Sciences, Peking University, Beijing
  • Xi Chen Stern School of Business, New York University
  • Qingming Huang University of Chinese Academy of Sciences, Beijing; IIP., Inst. of Comput. Tech., CAS, Beijing
  • Yuan Yao Hong Kong University of Science and Technology; BICMR and School of Mathematical Sciences, Peking University, Beijing, China

Abstract

Recently, crowdsourcing has emerged as an effective paradigm for human-powered large scale problem solving in various domains. However, task requester usually has a limited amount of budget, thus it is desirable to have a policy to wisely allocate the budget to achieve better quality. In this paper, we study the principle of information maximization for active sampling strategies in the framework of HodgeRank, an approach based on Hodge Decomposition of pairwise ranking data with multiple workers. The principle exhibits two scenarios of active sampling: Fisher information maximization that leads to unsupervised sampling based on a sequential maximization of graph algebraic connectivity without considering labels; and Bayesian information maximization that selects samples with the largest information gain from prior to posterior, which gives a supervised sampling involving the labels collected. Experiments show that the proposed methods boost the sampling efficiency as compared to traditional sampling schemes and are thus valuable to practical crowdsourcing experiments.

Downloads

Published

2018-04-29

How to Cite

Xu, Q., Xiong, J., Chen, X., Huang, Q., & Yao, Y. (2018). HodgeRank With Information Maximization for Crowdsourced Pairwise Ranking Aggregation. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/11619