RAFFMAN: Measuring and Analyzing Sentiment in Online Political Forum Discussions with an Application to the Trump Impeachment

Abstract

Given an online forum, how can we quantify changes in user affect towards a person or an idea over time? We argue that online political forums constitute an untapped opportunity for understanding sentiment toward aspects under discussion. However, the analysis of such forums has received little attention from the research community. In this paper, we develop RAFFMAN, a systematic approach to quantify the impact of external events on the affect of forum users towards a concept, such as a person or an entity. First, we develop an approach to capture and quantify the observed activity: we identify related keywords, filter threads, and establish correlations between events and spikes in the activity. Second, we modify and evaluate state-of-the-art NLP techniques to achieve high accuracy (74%) in a three-class sentiment classification problem. As a case study, we deploy our method to quantify the effect of President Trump’s impeachment on several concepts including: President Trump, Speaker Pelosi, and QAnon. Our data consists of 32M posts from Reddit and 4chan over a span of 6 months from September 2019 to February 2020. This initial analysis hints at an increase in political polarization, especially for people’s affect towards the President. Overall, our work is a building block towards mining the affect of online forum user towards a concept, which constitutes a untapped, massive, and publicly-available source of information.