Hate Cannot Drive Out Hate: Forecasting Conversation Incivility following Replies to Hate Speech

Abstract

User-generated counter hate speech is a promising means to combat hate speech, but questions about whether it can stop incivility in follow-up conversations linger. We argue that effective counter hate speech stops incivility from emerging in follow-up conversations—counter hate that elicits more incivility is counterproductive. This study introduces the task of predicting the incivility of conversations following replies to hate speech. We first propose a metric to measure conversation incivility based on the number of civil and uncivil comments as well as the unique authors involved in the discourse. Our metric approximates human judgments more accurately than previous metrics. We then use the metric to evaluate the outcomes of replies to hate speech. A linguistic analysis uncovers the differences in the language of replies that elicit follow-up conversations with high and low incivility. Experimental results show that forecasting incivility is challenging. We close with a qualitative analysis shedding light into the most common errors made by the best model.