From Compliance to Contextual Legitimacy: A Sociotechnical Framework for Evaluating AI Systems

Abstract

AI systems are increasingly recognized as sociotechnical systems whose impacts emerge not only from algorithmic behavior, but also from the contexts in which they operate. While Trustworthy AI (TAI) provides a foundational framework for AI governance, several sociotechnical legitimacy conditions remain difficult to systematically evaluate within predominantly artifact- and compliance-oriented approaches. This paper proposes a relational framework for evaluating contextual legitimacy in AI-enabled sociotechnical systems as a complement to conventional TAI-oriented evaluations. The framework identifies and operationalizes sociotechnical legitimacy conditions that remain fragmented, implicit, or weakly addressed within existing trustworthy AI approaches, and empirically applies them to TAI-oriented Urban Digital Twin literature. The findings suggest that complementing TAI with a relational sociotechnical perspective is necessary for more comprehensively evaluating the legitimacy of contemporary AI deployments.