Agentic Relationship Harm: Benchmarking and Gating Relational Manipulation in AI Agents

Authors

  • Tan Pei Sze National Institute of Informatics
  • Tasuku Igarashi Nagoya University
  • Isao Echizen National Institute of Informatics The University of Tokyo

DOI:

https://doi.org/10.1609/aies.v9i3.43289

Abstract

AI agents built on large language models can assist not only legitimate tasks but also relational manipulation. AI agents can be used to help a user maintain a deceptive identity, intensify emotional dependency, isolate a target, or prepare for later extraction. We conceptualise this risk as agentic relationship harm: workflow-level assistance that can exploit recipient vulnerability, persuasive influence, and relational power asymmetry. Existing safety evaluations and generic guardrails often treat harmfulness as a property of isolated outputs, missing role-sensitive interaction patterns. To study this, we introduce a 110-prompt benchmark with balanced attacker-and victim-side cases, a relationship-specific labelling framework, and a lightweight post-generation policy gate for local agent deployments. In our evaluation, the relationship-specific gate outperforms generic safety prompting under automated judging, with no harmful-compliance cases identified on the main benchmark or multi-turn stress test while preserving victim-side protective intervention. These results suggest that relationship harm is a distinct sociotechnical risk surface and that role-sensitive evaluation plus lightweight policy gating offers a practical path beyond generic refusal prompting.

Downloads

Published

2026-10-11

How to Cite

Pei Sze, T., Igarashi, T., & Echizen, I. (2026). Agentic Relationship Harm: Benchmarking and Gating Relational Manipulation in AI Agents. Proceedings of the AAAI ACM Conference on AI, Ethics, and Society, 9(3), 2065–2077. https://doi.org/10.1609/aies.v9i3.43289