Do Feature Attribution Methods Correctly Attribute Features?
DOI:
https://doi.org/10.1609/aaai.v36i9.21196Keywords:
Philosophy And Ethics Of AI (PEAI), Computer Vision (CV), Speech & Natural Language Processing (SNLP)Abstract
Feature attribution methods are popular in interpretable machine learning. These methods compute the attribution of each input feature to represent its importance, but there is no consensus on the definition of "attribution", leading to many competing methods with little systematic evaluation, complicated in particular by the lack of ground truth attribution. To address this, we propose a dataset modification procedure to induce such ground truth. Using this procedure, we evaluate three common methods: saliency maps, rationales, and attentions. We identify several deficiencies and add new perspectives to the growing body of evidence questioning the correctness and reliability of these methods applied on datasets in the wild. We further discuss possible avenues for remedy and recommend new attribution methods to be tested against ground truth before deployment. The code and appendix are available at https://yilunzhou.github.io/feature-attribution-evaluation/.Downloads
Published
2022-06-28
How to Cite
Zhou, Y., Booth, S., Ribeiro, M. T., & Shah, J. (2022). Do Feature Attribution Methods Correctly Attribute Features?. Proceedings of the AAAI Conference on Artificial Intelligence, 36(9), 9623-9633. https://doi.org/10.1609/aaai.v36i9.21196
Issue
Section
AAAI Technical Track on Philosophy and Ethics of AI