Shifting Gatekeeping Practices and Audience Engagement Under AI-Assisted Comment Moderation

Abstract

Traditionally, The New York Times has maintained substantial gatekeeping control over audience contributions on its website by selectively enabling comment sections on certain articles and pre-moderating comments. This study explores changes to these gatekeeping practices and subsequent audience engagement following the transition to AI-assisted comment moderation. The introduction of AI-assisted comment moderation coincided with an increase in articles with comment sections. However, we found that this increase was less pronounced for contentious, social issue topics than for soft news sections. Comparing comment quality before and after AI assistance across eight dimensions such as coherence and substantiveness, we observed some section-wise differences, although overall, the differences were minimal. We also found that AI-assisted moderation was associated with an increase in engagement among prolific commenters and an increase in the proportion of moderately active commenters. Overall, these findings indicate that AI-assisted moderation is associated with changes in gatekeeping and increased audience participation without compromising on the aggregate quality of comments published.