TY - JOUR AU - Oh, Changyong AU - Adamczewski, Kamil AU - Park, Mijung PY - 2020/04/03 Y2 - 2024/03/28 TI - Radial and Directional Posteriors for Bayesian Deep Learning JF - Proceedings of the AAAI Conference on Artificial Intelligence JA - AAAI VL - 34 IS - 04 SE - AAAI Technical Track: Machine Learning DO - 10.1609/aaai.v34i04.5976 UR - https://ojs.aaai.org/index.php/AAAI/article/view/5976 SP - 5298-5305 AB - <p>We propose a new variational family for Bayesian neural networks. We decompose the variational posterior into two components, where the <em>radial</em> component captures the strength of each neuron in terms of its magnitude; while the <em>directional</em> component captures the statistical dependencies among the weight parameters. The dependencies learned via the directional density provide better modeling performance compared to the widely-used Gaussian mean-field-type variational family. In addition, the strength of input and output neurons learned via our posterior provides a structured way to compress neural networks. Indeed, experiments show that our variational family improves predictive performance and yields compressed networks simultaneously.</p> ER -