Feature-Level Debiased Natural Language Understanding


  • Yougang Lyu Shandong University
  • Piji Li Nanjing University of Aeronautics and Astronautics
  • Yechang Yang Shandong University
  • Maarten de Rijke University of Amsterdam
  • Pengjie Ren Shandong University
  • Yukun Zhao Baidu Shandong University
  • Dawei Yin Baidu
  • Zhaochun Ren Shandong University




SNLP: Adversarial Attacks & Robustness, SNLP: Sentence-Level Semantics and Textual Inference, SNLP: Text Classification


Natural language understanding (NLU) models often rely on dataset biases rather than intended task-relevant features to achieve high performance on specific datasets. As a result, these models perform poorly on datasets outside the training distribution. Some recent studies address this issue by reducing the weights of biased samples during the training process. However, these methods still encode biased latent features in representations and neglect the dynamic nature of bias, which hinders model prediction. We propose an NLU debiasing method, named debiasing contrastive learning (DCT), to simultaneously alleviate the above problems based on contrastive learning. We devise a debiasing, positive sampling strategy to mitigate biased latent features by selecting the least similar biased positive samples. We also propose a dynamic negative sampling strategy to capture the dynamic influence of biases by employing a bias-only model to dynamically select the most similar biased negative samples. We conduct experiments on three NLU benchmark datasets. Experimental results show that DCT outperforms state-of-the-art baselines on out-of-distribution datasets while maintaining in-distribution performance. We also verify that DCT can reduce biased latent features from the model's representation.




How to Cite

Lyu, Y., Li, P., Yang, Y., de Rijke, M., Ren, P., Zhao, Y., Yin, D., & Ren, Z. (2023). Feature-Level Debiased Natural Language Understanding. Proceedings of the AAAI Conference on Artificial Intelligence, 37(11), 13353-13361. https://doi.org/10.1609/aaai.v37i11.26567



AAAI Technical Track on Speech & Natural Language Processing