Exploring the Music Perception Skills of Crowd Workers


  • Ioannis Petros Samiotis Delft University of Technology
  • Sihang Qiu Delft University of Technology
  • Christoph Lofi Delft University of Technology
  • Jie Yang Delft University of Technology
  • Ujwal Gadiraju Delft University of Technology
  • Alessandro Bozzon Delft University of Technology


Crowdsourcing, Music Annotation, Music Perception Skills, User Modelling, Audio Annotation, Music Sophistication, Human Computation


Music content annotation campaigns are common on paid crowdsourcing platforms. Crowd workers are expected to annotate complicated music artefacts, which can demand certain skills and expertise. Traditional methods of participant selection are not designed to capture these kind of domain-specific skills and expertise, and often domain-specific questions fall under the general demographics category. Despite the popularity of such tasks, there is a general lack of deeper understanding of the distribution of musical properties - especially auditory perception skills - among workers. To address this knowledge gap, we conducted a user study (N=100) on Prolific. We asked workers to indicate their musical sophistication through a questionnaire and assessed their music perception skills through an audio-based skill test. The goal of this work is to better understand the extent to which crowd workers possess higher perceptions skills, beyond their own musical education level and self reported abilities. Our study shows that untrained crowd workers can possess high perception skills on the music elements of melody, tuning, accent and tempo; skills that can be useful in a plethora of annotation tasks in the music domain.




How to Cite

Samiotis, I. P., Qiu, S., Lofi, C., Yang, J., Gadiraju, U., & Bozzon, A. (2021). Exploring the Music Perception Skills of Crowd Workers. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 9(1), 108-119. Retrieved from https://ojs.aaai.org/index.php/HCOMP/article/view/18944