Backforward Propagation (Student Abstract) - Stoica, George; Simionescu, Cristian (Proceedings of the AAAI Conference on Artificial Intelligence, 2024)
In this paper we introduce Backforward Propagation, a method of completely eliminating Internal Covariate Shift (ICS). Unlike previous methods, which only indirectly reduce the impact of ICS while introducing other biases, we are able to have a surgical view at the effects ICS has on training neural networks. Our experiments show that ICS has a weight regularizing effect on models, and completely removing it enables for faster convergence of the neural network.