Training Spiking Neural Networks with Accumulated Spiking Flow

Authors

  • Hao Wu University of Science and Technology of China
  • Yueyi Zhang University of Science and Technology of China
  • Wenming Weng University of Science and Technology of China
  • Yongting Zhang University of Science and Technology of China
  • Zhiwei Xiong University of Science and Technology of China
  • Zheng-Jun Zha University of Science and Technology of China
  • Xiaoyan Sun University of Science and Technology of China
  • Feng Wu University of Science and Technology of China

DOI:

https://doi.org/10.1609/aaai.v35i12.17236

Keywords:

(Deep) Neural Network Algorithms

Abstract

The fast development of neuromorphic hardwares promotes Spiking Neural Networks (SNNs) to a thrilling research avenue. Current SNNs, though much efficient, are less effective compared with leading Artificial Neural Networks (ANNs) especially in supervised learning tasks. Recent efforts further demonstrate the potential of SNNs in supervised learning by introducing approximated backpropagation (BP) methods. To deal with the non-differentiable spike function in SNNs, these BP methods utilize information from the spatio-temporal domain to adjust the model parameters. With the increasing of time window and network size, the computational complexity of spatio-temporal backpropagation augments dramatically. In this paper, we propose a new backpropagation method for SNNs based on the accumulated spiking flow (ASF), i.e. ASF-BP. In the proposed ASF-BP method, updating parameters does not rely on the spike train of spiking neurons but leverage accumulated inputs and outputs of spiking neurons over the time window, which reduces the BP complexity significantly. We further present an adaptive linear estimation model to approach the dynamic characteristics of spiking neurons statistically. Experimental results demonstrate that with our proposed ASF-BP method, light-weight convolutional SNNs achieve superior performances compared with other spike-based BP methods on both non-neuromorphic (MNIST, CIFAR10) and neuromorphic (CIFAR10-DVS) datasets. The code is available at https://github.com/neural-lab/ASF-BP.

Downloads

Published

2021-05-18

How to Cite

Wu, H., Zhang, Y., Weng, W., Zhang, Y., Xiong, Z., Zha, Z.-J., Sun, X., & Wu, F. (2021). Training Spiking Neural Networks with Accumulated Spiking Flow. Proceedings of the AAAI Conference on Artificial Intelligence, 35(12), 10320-10328. https://doi.org/10.1609/aaai.v35i12.17236

Issue

Section

AAAI Technical Track on Machine Learning V