Curse or Redemption? How Data Heterogeneity Affects the Robustness of Federated Learning

Authors

  • Syed Zawad University of Nervada, Reno
  • Ahsan Ali University of Nevada, Reno
  • Pin-Yu Chen IBM Research
  • Ali Anwar IBM Research
  • Yi Zhou IBM Research
  • Nathalie Baracaldo IBM Research
  • Yuan Tian University of Virginia
  • Feng Yan University of Nevada, Reno

Keywords:

Distributed Machine Learning & Federated Learning, Adversarial Attacks & Robustness

Abstract

Data heterogeneity has been identified as one of the key features in federated learning but often overlooked in the lens of robustness to adversarial attacks. This paper focuses on characterizing and understanding its impact on backdooring attacks in federated learning through comprehensive experiments using synthetic and the LEAF benchmarks. The initial impression driven by our experimental results suggests that data heterogeneity is the dominant factor in the effectiveness of attacks and it may be a redemption for defending against backdooring as it makes the attack less efficient, more challenging to design effective attack strategies, and the attack result also becomes less predictable. However, with further investigations, we found data heterogeneity is more of a curse than a redemption as the attack effectiveness can be significantly boosted by simply adjusting the client-side backdooring timing. More importantly, data heterogeneity may result in overfitting at the local training of benign clients, which can be utilized by attackers to disguise themselves and fool skewed-feature based defenses. In addition, effective attack strategies can be made by adjusting attack data distribution. Finally, we discuss the potential directions of defending the curses brought by data heterogeneity. The results and lessons learned from our extensive experiments and analysis offer new insights for designing robust federated learning methods and systems.

Downloads

Published

2021-05-18

How to Cite

Zawad, S., Ali, A., Chen, P.-Y., Anwar, A., Zhou, Y., Baracaldo, N., Tian, Y., & Yan, F. (2021). Curse or Redemption? How Data Heterogeneity Affects the Robustness of Federated Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 35(12), 10807-10814. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17291

Issue

Section

AAAI Technical Track on Machine Learning V