Does the Geometry of the Data Control the Geometry of Neural Predictions? (Student Abstract)
Keywords:Deep Learning, Information Theory, Fisher Information Matrix
AbstractThis paper studies the over-parameterization of deep neural networks using the Fisher Information Matrix from information geometry. We identify several surprising trends in the structure of its eigenspectrum, and how this structure relates to the eigenspectrum of the data correlation matrix. We identify how the eigenspectrum relates to the topology of the predictions of the model and develop a "model reduction'' method for deep networks. This ongoing investigation hypothesizes certain universal trends in the FIM of deep networks that may shed light on their effectiveness.
How to Cite
Cowlagi, A., & Chaudhari, P. (2022). Does the Geometry of the Data Control the Geometry of Neural Predictions? (Student Abstract). Proceedings of the AAAI Conference on Artificial Intelligence, 36(11), 12931-12932. https://doi.org/10.1609/aaai.v36i11.21602
AAAI Student Abstract and Poster Program