From Oracles to Productive Friction: Designing for Meaningful Human Oversight in Clinical AI

Abstract

We propose productive friction as a design principle for clinical AI: the deliberate, graded introduction of interaction cost when accepting machine recommendations, calibrated to case uncertainty so that analytical reasoning is activated without disrupting workflow. We ground this principle in a mixed-method study with eleven practising radiologists interacting with two contrasting prototypes, an oracular system delivering a single diagnosis and an evaluative system presenting multiple differentials with reference exemplars. Results reveal a consistent preference for evaluative support in complex cases, where it enhances diagnostic reasoning, confidence, and reflective engagement, while oracular outputs were favoured for routine tasks. Radiologists described evaluative support as "accompanying" their thinking, preserving professional agency and offering pedagogical value. We argue that productive friction carries direct ethical significance: when agreeing with an AI recommendation requires less effort than disagreeing, the clinician's role risks becoming procedural rather than a genuine locus of judgement. We operationalise this principle along four dimensions, temporal delay, option cardinality, evidence depth, and commitment structure, and translate it into implementable design mechanisms for adaptive clinical AI.