From Human Explanation to Model Interpretability: A Framework Based on Weight of Evidence

Authors

  • David Alvarez Melis Microsoft Research
  • Harmanpreet Kaur University of Michigan
  • Hal Daumé III Microsoft Research University of Maryland
  • Hanna Wallach Microsoft Research
  • Jennifer Wortman Vaughan Microsoft Research

Keywords:

Interpretability, Explanation, Weight Of Evidence, Human-centered

Abstract

We take inspiration from the study of human explanation to inform the design and evaluation of interpretability methods in machine learning. First, we survey the literature on human explanation in philosophy, cognitive science, and the social sciences, and propose a list of design principles for machine-generated explanations that are meaningful to humans. Using the concept of weight of evidence from information theory, we develop a method for generating explanations that adhere to these principles. We show that this method can be adapted to handle high-dimensional, multi-class settings, yielding a flexible framework for generating explanations. We demonstrate that these explanations can be estimated accurately from finite samples and are robust to small perturbations of the inputs. We also evaluate our method through a qualitative user study with machine learning practitioners, where we observe that the resulting explanations are usable despite some participants struggling with background concepts like prior class probabilities. Finally, we conclude by surfacing design implications for interpretability tools in general.

Downloads

Published

2021-10-04

How to Cite

Alvarez Melis, D., Kaur, H., Daumé III, H., Wallach, H., & Wortman Vaughan, J. (2021). From Human Explanation to Model Interpretability: A Framework Based on Weight of Evidence. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 9(1), 35-47. Retrieved from https://ojs.aaai.org/index.php/HCOMP/article/view/18938