Distribution Matching for Rationalization

Authors

  • Yongfeng Huang Tsinghua University, Beijing
  • Yujun Chen Recurrent AI
  • Yulun Du Recurrent AI
  • Zhilin Yang Recurrent AI

Keywords:

Interpretaility & Analysis of NLP Models

Abstract

The task of rationalization aims to extract pieces of input text as rationales to justify neural network predictions on text classification tasks. By definition, rationales represent key text pieces used for prediction and thus should have similar classification feature distribution compared to the original input text. However, previous methods mainly focused on maximizing the mutual information between rationales and labels while neglecting the relationship between rationales and input text. To address this issue, we propose a novel rationalization method that matches the distributions of rationales and input text in both the feature space and output space. Empirically, the proposed distribution matching approach consistently outperforms previous methods by a large margin. Our data and code are available.

Downloads

Published

2021-05-18

How to Cite

Huang, Y., Chen, Y., Du, Y., & Yang, Z. (2021). Distribution Matching for Rationalization. Proceedings of the AAAI Conference on Artificial Intelligence, 35(14), 13090-13097. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17547

Issue

Section

AAAI Technical Track on Speech and Natural Language Processing I