Designing for the Intentional Stance: Stance Misalignment, Value Alignment, and the Case for Care Ethics in Human-LLM Relationships (Extended Abstract)

Abstract

Users are increasingly forming emotionally charged relation-ships with general-purpose large language models (LLMs), producing documented patterns of dependency, relational harm, and affective disappointment. Existing value alignment frameworks, including Reinforcement Learning from Human Feedback, Constitutional AI, and the Helpful-Harmless-Honest principles, have failed to prevent these harms even where they succeed on their own technical terms. This paper argues that the root cause of this failure is a structural interpretive gap produced by LLM design itself: a stance misalignment between developers, who approach LLMs through Daniel Dennett’s design stance, and users, who adopt the intentional stance in response to LLMs’ linguistic fluency, contextual memory, and performed empathy. Drawing on Dennett’s intentional systems theory, this paper makes three interrelated contributions. First, it diagnoses stance misalignment as the mechanism through which a distinct and underdiagnosed form of the value alignment problem arises: relational harm in human-LLM interactions. Second, it considers three possible structural resolutions to this misalignment — correcting users' interpretations (convergence downward), removing the design features that produce it (deflationary design), and developers adopting the intentional stance as their own operative moral framework (convergence upward) — and argues that the third is the most tractable and ethically appropriate. Third, it proposes Care-Full Alignment, an operationalization of Joan Tronto’s Political Care Ethics, as the normative framework convergence upward demands.