AI-Mediated Dispute Resolution

Abstract

We examine the effectiveness of large language model (LLM) mediations in the under-studied dispute resolution domain. We first used a new corpus of dispute resolutions, KODIS, to investigate if LLMs can correctly identify whether to intervene. We find evidence that GPT as a mediator picks up on salient aspects of a dispute, such as Frustration and whether the disputants ultimately come to a resolution or stall at an impasse --- intervening significantly more so in cases of high frustration and impasse. Afterward, we ran a user study to compare GPT mediations against those of novice human mediators. We find participants agreed GPT's mediations were more likely to lead to resolution; were better positioned in the dialog; had better justification than human-crafted ones; and, on a forced choice, were generally more effective than novice human mediations.