Crowdsourcing Complex Workflows under Budget Constraints


  • Long Tran-Thanh University of Southampton
  • Trung Dong Huynh University of Southampton
  • Avi Rosenfeld Jerusalem College of Technology
  • Sarvapali Ramchurn University of Southampton
  • Nicholas Jennings University of Southampton



crowdsourcing, budget constraints, complex workflow, task allocation


We consider the problem of task allocation in crowdsourcing systems with multiple complex workflows, each of which consists of a set of inter-dependent micro-tasks.We propose Budgeteer, an algorithm to solve this problem under a budget constraint. In particular, our algorithm first calculates an efficient way to allocate budget to each workflow. It then determines the number of inter-dependent micro-tasks and the price to pay for each task within each workflow, given the corresponding budget constraints. We empirically evaluate it on a well-known crowdsourcing-based text correction workflow using Amazon Mechanical Turk, and show that Budgeteer can achieve similar levels of accuracy to current benchmarks, but is on average 45 % cheaper.




How to Cite

Tran-Thanh, L., Huynh, T. D., Rosenfeld, A., Ramchurn, S., & Jennings, N. (2015). Crowdsourcing Complex Workflows under Budget Constraints. Proceedings of the AAAI Conference on Artificial Intelligence, 29(1).



AAAI Technical Track: Human-Computation and Crowd Sourcing