Reliability Analysis of Psychological Concept Extraction and Classification in User-Penned Text


  • Muskan Garg Mayo Clinic
  • MSVPJ Sathvik IIIT Dharwad
  • Shaina Raza Vector Institute
  • Amrit Chadha Indian Institute of Technology
  • Sunghwan Sohn Mayo Clinic



The social NLP research community witness a recent surge in the computational advancements of mental health analysis to build responsible AI models for a complex interplay between language use and self-perception. Such responsible AI models aid in quantifying the psychological concepts from user-penned texts on social media. On thinking beyond the low-level (classification) task, we advance the existing binary classification dataset, towards a higher-level task of reliability analysis through the lens of explanations, posing it as one of the safety measures. We annotate the LoST dataset to capture nuanced textual cues that suggest the presence of low self-esteem in the posts of Reddit users. We further state that the NLP models developed for determining the presence of low self-esteem, focus more on three types of textual cues: (i) Trigger: words that triggers mental disturbance, (ii) LoST indicators: text indicators emphasizing low self-esteem, and (iii) Consequences: words describing the consequences of mental disturbance. We implement existing classifiers to examine the attention mechanism in pre-trained language models (PLMs) for a domain-specific psychology-grounded task. Our findings suggest the need of shifting the focus of PLMs from Trigger and Consequences to a more comprehensive explanation, emphasizing LoST indicators while determining low self-esteem in Reddit posts.




How to Cite

Garg, M., Sathvik, M., Raza, S., Chadha, A., & Sohn, S. (2024). Reliability Analysis of Psychological Concept Extraction and Classification in User-Penned Text. Proceedings of the International AAAI Conference on Web and Social Media, 18(1), 422-434.