Learning Continuous High-Dimensional Models using Mutual Information and Copula Bayesian Networks

Authors

  • Marvin Lasserre Laboratoire d'Informatique de Paris 6
  • Régis Lebrun Airbus Central Research & Technology
  • Pierre-Henri Wuillemin Laboratoire d'Informatique de Paris 6

Keywords:

Bayesian Networks, Graphical Models

Abstract

We propose a new framework to learn non-parametric graphical models from continuous observational data. Our method is based on concepts from information theory in order to discover independences and causality between variables: the conditional and multivariate mutual information (such as \cite{verny2017learning} for discrete models). To estimate these quantities, we propose non-parametric estimators relying on the Bernstein copula and that are constructed by exploiting the relation between the mutual information and the copula entropy \cite{ma2011mutual, belalia2017testing}. To our knowledge, this relation is only documented for the bivariate case and, for the need of our algorithms, is here extended to the conditional and multivariate mutual information. This framework leads to a new algorithm to learn continuous non-parametric Bayesian network. Moreover, we use this estimator to speed up the BIC algorithm proposed in \cite{elidan2010copula} by taking advantage of the decomposition of the likelihood function in a sum of mutual information \cite{koller2009probabilistic}. Finally, our method is compared in terms of performances and complexity with other state of the art techniques to learn Copula Bayesian Networks and shows superior results. In particular, it needs less data to recover the true structure and generalizes better on data that are not sampled from Gaussian distributions.

Downloads

Published

2021-05-18

How to Cite

Lasserre, M., Lebrun, R., & Wuillemin, P.-H. (2021). Learning Continuous High-Dimensional Models using Mutual Information and Copula Bayesian Networks. Proceedings of the AAAI Conference on Artificial Intelligence, 35(13), 12139-12146. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17441

Issue

Section

AAAI Technical Track on Reasoning under Uncertainty