Variational Disentanglement for Rare Event Modeling

Authors

  • Zidi Xiu Duke University
  • Chenyang Tao Duke University
  • Michael Gao Duke University
  • Connor Davis Duke Institute for Health Innovation
  • Benjamin A. Goldstein Duke University
  • Ricardo Henao Duke University

DOI:

https://doi.org/10.1609/aaai.v35i12.17253

Keywords:

Multi-class/Multi-label Learning & Extreme Classification, AI Responses to the COVID-19 Pandemic (Covid19)

Abstract

Combining the increasing availability and abundance of healthcare data and the current advances in machine learning methods have created renewed opportunities to improve clinical decision support systems. However, in healthcare risk prediction applications, the proportion of cases with the condition (label) of interest is often very low relative to the available sample size. Though very prevalent in healthcare, such imbalanced classification settings are also common and challenging in many other scenarios. So motivated, we propose a variational disentanglement approach to semi-parametrically learn from rare events in heavily imbalanced classification problems. Specifically, we leverage the imposed extreme-distribution behavior on a latent space to extract information from low-prevalence events, and develop a robust prediction arm that joins the merits of the generalized additive model and isotonic neural nets. Results on synthetic studies and diverse real-world datasets, including mortality prediction on a COVID-19 cohort, demonstrate that the proposed approach outperforms existing alternatives.

Downloads

Published

2021-05-18

How to Cite

Xiu, Z., Tao, C., Gao, M., Davis, C., Goldstein, B. A., & Henao, R. (2021). Variational Disentanglement for Rare Event Modeling. Proceedings of the AAAI Conference on Artificial Intelligence, 35(12), 10469-10477. https://doi.org/10.1609/aaai.v35i12.17253

Issue

Section

AAAI Technical Track on Machine Learning V