Privacy Cost as Equity Input: A Group Fairness Criterion for Differentially Private Machine Learning

Abstract

Differential privacy (DP) is increasingly deployed to limit membership inference risk in machine-learning systems. Prior work has shown that DP-SGD can widen accuracy disparities across demographic groups, but this framing treats fairness as a purely outcome-side concern. We argue that privacy cost (denoted as the information leakage borne by each group) is itself a form of harm: a group that involuntarily bears greater privacy exposure is entitled to proportionally greater benefit from the system. From this principle we derive the Privacy-Cost Equity Ratio (PCER), a group fairness metric defined as a group’s positive prediction rate normalized by its per-group overfitting gap. By a standard membership inference bound, this overfitting gap upper-bounds each group’s advantage under the canonical threshold attack, making PCER a conservative measure of benefit relative to exposure. PCER requires only per-group train and test accuracy (no shadow models required) making it a practical post-hoc audit tool. We evaluate PCER alongside standard fairness metrics across six benchmark-attribute combinations spanning tabular and NLP domains, under DP-SGD at a range of privacy budgets. The results reveal patterns that outcome-based metrics miss. On Adult Income (race), PCER inverts the fairness ranking relative to standard metrics at the default floor constant: the privacy budget deemed most equitable by outcome gaps is not the most equitable by exposure-adjusted ones, though this specific ranking is itself sensitive to the floor constant. On COMPAS, PCER uncovers a persistently lower benefit-to-cost ratio for the protected group that demographic parity gap masks entirely. Sensitivity analysis further shows that very strong privacy guarantees collapse both groups’ overfitting to a numerical floor, rendering exposure-based audits uninformative in that regime. Taken together, these findings argue that fairness audits of privacy-preserving systems must account for who bears the cost of protection, as well as who benefits from its outcomes.