TRIED: Truly Innovative and Effective AI Detection Benchmark

Abstract

The rapid rise of generative AI and the increasing ability to create deceptive synthetic media pose a significant threat to public trust and information integrity, particularly in resource-constrained regions of the Global Majority. In the face of this growth, AI detection tools remain an invaluable resource for frontline information actors, but continue to underperform in real-world scenarios due to challenges related to explainability, fairness, accessibility, and contextual relevance. Drawing on frontline experiences, deceptive AI cases, and global consultations, this paper highlights the limitations of current AI detection tools and outlines how they must evolve to become truly innovative and relevant by meeting diverse linguistic, cultural, and technological contexts. It also introduces the Truly Innovative and Effective AI Detection (TRIED) Benchmark, a new framework for evaluating detection tools based on their real-world impact and capacity for innovation to guide inclusive AI detection tools development as well as support policy efforts towards AI procedures and standards reflecting global frontline community input.