Human Uncertainty Inference via Deterministic Ensemble Neural Networks

Authors

  • Yujin Cha Department of Bio and Brain Engineering, Korea Advanced Institute of Science and Technology (KAIST)
  • Sang Wan Lee Department of Bio and Brain Engineering, Korea Advanced Institute of Science and Technology (KAIST) Program of Brain and Cognitive Engineering, Korea Advanced Institute of Science and Technology (KAIST) Center for Neuroscience-inspired Artificial Intelligence, Korea Advanced Institute of Science and Technology (KAIST)

Keywords:

Learning Human Values and Preferences, Human-in-the-loop Machine Learning, Calibration & Uncertainty Quantification, Healthcare, Medicine & Wellness

Abstract

The estimation and inference of human predictive uncertainty have great potential to improve the sampling efficiency and prediction reliability of human-in-the-loop systems for smart healthcare, smart education, and human-computer interactions. Predictive uncertainty in humans is highly interpretable, but its measurement is poorly accessible. Contrarily, the predictive uncertainty of machine learning models, albeit with poor interpretability, is relatively easily accessible. Here, we demonstrate that the poor accessibility of human uncertainty can be resolved by exploiting simple and universally accessible deterministic neural networks. We propose a new model for human uncertainty inference, called proxy ensemble network (PEN). Simulations with a few benchmark datasets demonstrated that the model can efficiently learn human uncertainty from a small amount of data. To show its applicability in real-world problems, we performed behavioral experiments, in which 64 physicians classified medical images and reported their level of confidence. We showed that the PEN could predict both the uncertainty range and diagnoses given by subjects with high accuracy. Our results demonstrate the ability of machine learning in guiding human decision making; it can also help humans in learning more efficiently and accurately. To the best of our knowledge, this is the first study that explored the possibility of accessing human uncertainty via the lens of deterministic neural networks.

Downloads

Published

2021-05-18

How to Cite

Cha, Y., & Lee, S. W. (2021). Human Uncertainty Inference via Deterministic Ensemble Neural Networks. Proceedings of the AAAI Conference on Artificial Intelligence, 35(7), 5877-5886. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/16735

Issue

Section

AAAI Technical Track on Humans and AI