Real or Fake Text?: Investigating Human Ability to Detect Boundaries between Human-Written and Machine-Generated Text

Authors

  • Liam Dugan University of Pennsylvania
  • Daphne Ippolito University of Pennsylvania
  • Arun Kirubarajan University of Pennsylvania
  • Sherry Shi University of Pennsylvania
  • Chris Callison-Burch University of Pennsylvania

DOI:

https://doi.org/10.1609/aaai.v37i11.26501

Keywords:

SNLP: Interpretability & Analysis of NLP Models, SNLP: Language Models, HAI: Learning Human Values and Preferences, SNLP: Generation, SNLP: Bias, Fairness, Transparency & Privacy

Abstract

As text generated by large language models proliferates, it becomes vital to understand how humans engage with such text, and whether or not they are able to detect when the text they are reading did not originate with a human writer. Prior work on human detection of generated text focuses on the case where an entire passage is either human-written or machine-generated. In this paper, we study a more realistic setting where text begins as human-written and transitions to being generated by state-of-the-art neural language models. We show that, while annotators often struggle at this task, there is substantial variance in annotator skill and that given proper incentives, annotators can improve at this task over time. Furthermore, we conduct a detailed comparison study and analyze how a variety of variables (model size, decoding strategy, fine-tuning, prompt genre, etc.) affect human detection performance. Finally, we collect error annotations from our participants and use them to show that certain textual genres influence models to make different types of errors and that certain sentence-level features correlate highly with annotator selection. We release the RoFT dataset: a collection of over 21,000 human annotations paired with error classifications to encourage future work in human detection and evaluation of generated text.

Downloads

Published

2023-06-26

How to Cite

Dugan, L., Ippolito, D., Kirubarajan, A., Shi, S., & Callison-Burch, C. (2023). Real or Fake Text?: Investigating Human Ability to Detect Boundaries between Human-Written and Machine-Generated Text. Proceedings of the AAAI Conference on Artificial Intelligence, 37(11), 12763-12771. https://doi.org/10.1609/aaai.v37i11.26501

Issue

Section

AAAI Technical Track on Speech & Natural Language Processing