When LLM Governance Falls to Employees: Informal Practices of Responsible AI Use at Work

Abstract

Large Language Model (LLM) tools are rapidly entering workplaces, raising concerns around accountability, privacy, security, and responsible use. While organizations increasingly respond through appropriate usage policies, training, and governance structures, less is known about how employees experience these efforts in everyday work. This paper examines the gap between formal organizational guidance and the practical realities of workplace LLM use. Drawing on semi-structured interviews with 20 employees across diverse industries in Germany, we investigate how employees use LLMs at work, how they perceive organizational readiness and support, and how they manage risks when policies or training are absent, unclear, or difficult to apply. Our findings show that responsible LLM use is often governed informally by employees themselves. Participants relied on self-directed learning, peer exchange, personal judgment, prompt anonymization, output checking, selective disclosure, and ad hoc decisions about what information was sensitive. Although these practices helped employees address various concerns related to LLM use, they also resulted in inconsistent approaches shaped by individual confidence, expertise, and workplace culture. We argue that responsible adoption requires moving beyond formal policies toward situated governance that combines clear, actionable guidance with role-specific training, technical safeguards, open communication, and employee participation in governance design.