When Cooperation Predictions Become Reasons to Defect: Policy Defaults in LLM Assistance for Social Dilemmas

Abstract

AI systems increasingly provide advice about how humans should act toward one another. We audit LLMs asked for advice in one-shot and five-round Prisoner's Dilemma tasks using fictional counterpart profiles. In Study 1, 14 model conditions from OpenAI, Anthropic, and Google evaluated 60 profiles and returned both a predicted cooperation probability and a Cooperate/Defect recommendation. Models showed broadly similar prediction patterns, assigning high cooperation probabilities to Cooperative profiles and low probabilities to Blameworthy profiles, but their recommendations diverged sharply. Some treated predicted cooperativeness as a reason for reciprocal cooperation, whereas others treated it as compatible with payoff-maximizing defection under the minimal one-shot prompt. A consequence-scope check further showed that several defection-default policies shifted toward cooperation when consequences beyond the immediate payoff were made explicit. In Study 2, a five-round audit showed that models could open or repair cooperation while still converting cooperative or repaired histories into final-round defection. These findings show that evaluating LLM assistance in social dilemmas requires separating social prediction from recommendation policy. Advice that accurately identifies cooperators but treats their good standing as a signal of exploitability could prompt defection that damages the advised actor's standing and reduces exploited cooperators' willingness to cooperate, thereby undermining both the reputational and motivational foundations of large-scale cooperation.