The Hidden Cost of a Friendly Face: Investigating the Effect of Agent Embodiment on Trust Loss in Conversational Recommender Systems After Misleading Health Claims

Abstract

Conversational recommender systems (CRSs) are becoming pervasive technologies, supporting everyday decisions in domains ranging from e-commerce to health and dietary contexts. In these settings, there is a growing trend to deliver CRSs as embodied conversational agents (ECAs)–i.e. virtual agents with visible avatars, synthesised voices, and nonverbal behaviours that simulate face-to-face human interaction. Agent embodiment has been linked to greater engagement, perceived trustworthiness, and user reliance on agent recommendations. Yet, how embodiment shapes user responses when an agent’s recommendation is misleading remains underexplored. We argue this is an ethical blind spot with particular consequence in health contexts, where flawed recommendations may influence decisions with meaningful consequences for well-being. Drawing on literature from trust in human-AI interaction, recommender systems, and embodied conversational agents, we conducted a between-subjects experiment in a healthy recipe recommendation scenario to investigate how embodiment shapes trust loss along the dimensions of goodwill and qualification trust following a misleading health recommendation, and how any buffering cascades into users’ willingness to follow the recommendation and their intention to continue using the agent. The findings carry implications for the responsible design and deployment of embodied CRSs, particularly in health contexts.