Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior


  • Antigoni Founta Aristotle University of Thessaloniki
  • Constantinos Djouvas Cyprus University of Technology
  • Despoina Chatzakou Aristotle University of Thessaloniki
  • Ilias Leontiadis Telefonica Research
  • Jeremy Blackburn University of Alabama at Birmingham
  • Gianluca Stringhini University College London
  • Athena Vakali Aristotle University of Thessaloniki
  • Michael Sirivianos Cyprus University of Technology
  • Nicolas Kourtellis Telefonica Research



hate speech, twitter, abusive, crowdsourcing


In recent years online social networks have suffered an increase in sexism, racism, and other types of aggressive and cyberbullying behavior, often manifesting itself through offensive, abusive, or hateful language. Past scientific work focused on studying these forms of abusive activity in popular online social networks, such as Facebook and Twitter. Building on such work, we present an eight month study of the various forms of abusive behavior on Twitter, in a holistic fashion. Departing from past work, we examine a wide variety of labeling schemes, which cover different forms of abusive behavior. We propose an incremental and iterative methodology that leverages the power of crowdsourcing to annotate a large collection of tweets with a set of abuse-related labels. By applying our methodology and performing statistical analysis for label merging or elimination, we identify a reduced but robust set of labels to characterize abuse-related tweets. Finally, we offer a characterization of our annotated dataset of 80 thousand tweets, which we make publicly available for further scientific exploration.




How to Cite

Founta, A., Djouvas, C., Chatzakou, D., Leontiadis, I., Blackburn, J., Stringhini, G., Vakali, A., Sirivianos, M., & Kourtellis, N. (2018). Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior. Proceedings of the International AAAI Conference on Web and Social Media, 12(1).