Xu, Y.; Ye, X.; Chen, Y.; Zhang, Q. When Human Preferences Flip: An Instance-Dependent Robust Loss for RLHF. AAAI 2026, 40, 38057-38065.