Bridging the Knowledge Gap: Understanding User Expectations for Trustworthy LLM Standards

Abstract

Researchers, policymakers, and developers of artificial intelligence (AI) are actively collaborating to establish trustworthy AI standards that align with broader societal values, particularly in the context of large language models (LLMs). However, the critical discourse on bridging the vast knowledge gap between experts who shape and implement standards for LLMs and users whose values are at stake remains largely unaddressed. Taking a "bottom-up" perspective and using a mixed-method approach, we first conducted interviews (N = 12) to engage with users' perceptions of normative standards in the context of LLMs. We thereby identified 68 specific criteria that users' consider when evaluating whether their values are fulfilled. Second, we conducted an online survey (N = 379) to further investigate how users prioritize these standards and the identified criteria in conversational LLM-based applications. Our findings reveal opportunities for strategic communication measures, the importance of transparent governance mechanisms and the necessity of non-technical complements to technical solutions for bridging the knowledge gap. We discuss actionable steps to effectively communicate trustworthy AI standards.