Towards Emotion- and Time-Aware Classification of Tweets to Assist Human Moderation for Suicide Prevention


  • Ramit Sawhney Netaji Subhas Institute of Technology
  • Harshit Joshi Delhi University
  • Alicia Nobles University of California San Diego
  • Rajiv Ratn Shah IIIT Delhi



Social innovation and effecting change through social media, Human computer interaction; social media tools; navigation and visualization, Subjectivity in textual data; sentiment analysis; polarity/opinion identification and extraction, linguistic analyses of social media behavior, Psychological, personality-based and ethnographic studies of social media


Social media platforms are already engaged in leveraging existing online socio-technical systems to employ just-in-time interventions for suicide prevention to the public. These efforts primarily rely on self-reports of potential self-harm content that is reviewed by moderators. Most recently, platforms have employed automated models to identify self-harm content, but acknowledge that these automated models still struggle to understand the nuance of human language (e.g., sarcasm). By explicitly focusing on Twitter posts that could easily be misidentified by a model as expressing suicidal intent (i.e., they contain similar phrases such as ``wanting to die''), our work examines the temporal differences in historical expressions of general and emotional language prior to a clear expression of suicidal intent. Additionally, we analyze time-aware neural models that build on these language variants and factors in the historical, emotional spectrum of a user's tweeting activity. The strongest model achieves high (statistically significant) performance (macro F1=0.804, recall=0.813) to identify social media indicative of suicidal intent. Using three use cases of tweets with phrases common to suicidal intent, we qualitatively analyze and interpret how such models decided if suicidal intent was present and discuss how these analyses may be used to alleviate the burden on human moderators within the known constraints of how moderation is performed (e.g., no access to the user's timeline). Finally, we discuss the ethical implications of such data-driven models and inferences about suicidal intent from social media. Content warning: this article discusses self-harm and suicide.




How to Cite

Sawhney, R., Joshi, H., Nobles, A., & Shah, R. R. (2021). Towards Emotion- and Time-Aware Classification of Tweets to Assist Human Moderation for Suicide Prevention. Proceedings of the International AAAI Conference on Web and Social Media, 15(1), 609-620.