Word Embeddings via Causal Inference: Gender Bias Reducing and Semantic Information Preserving

Authors

  • Lei Ding University of Alberta
  • Dengdeng Yu University of Texas at Arlington
  • Jinhan Xie University of Alberta
  • Wenxing Guo University of Alberta
  • Shenggang Hu University of Essex
  • Meichen Liu University of Alberta
  • Linglong Kong University of Alberta
  • Hongsheng Dai University of Essex
  • Yanchun Bao University of Essex
  • Bei Jiang University of Alberta

DOI:

https://doi.org/10.1609/aaai.v36i11.21443

Keywords:

AI For Social Impact (AISI Track Papers Only)

Abstract

With widening deployments of natural language processing (NLP) in daily life, inherited social biases from NLP models have become more severe and problematic. Previous studies have shown that word embeddings trained on human-generated corpora have strong gender biases that can produce discriminative results in downstream tasks. Previous debiasing methods focus mainly on modeling bias and only implicitly consider semantic information while completely overlooking the complex underlying causal structure among bias and semantic components. To address these issues, we propose a novel methodology that leverages a causal inference framework to effectively remove gender bias. The proposed method allows us to construct and analyze the complex causal mechanisms facilitating gender information flow while retaining oracle semantic information within word embeddings. Our comprehensive experiments show that the proposed method achieves state-of-the-art results in gender-debiasing tasks. In addition, our methods yield better performance in word similarity evaluation and various extrinsic downstream NLP tasks.

Downloads

Published

2022-06-28

How to Cite

Ding, L., Yu, D., Xie, J., Guo, W., Hu, S., Liu, M., Kong, L., Dai, H., Bao, Y., & Jiang, B. (2022). Word Embeddings via Causal Inference: Gender Bias Reducing and Semantic Information Preserving. Proceedings of the AAAI Conference on Artificial Intelligence, 36(11), 11864-11872. https://doi.org/10.1609/aaai.v36i11.21443