Joint Demosaicing and Denoising for Spike Camera


  • Yanchen Dong Peking University
  • Ruiqin Xiong Peking University
  • Jing Zhao National Computer Network Emergency Response Technical Team Peking University
  • Jian Zhang Peking University
  • Xiaopeng Fan Harbin Institute of Technology
  • Shuyuan Zhu University of Electronic Science and Technology of China
  • Tiejun Huang Peking University



CV: Computational Photography, Image & Video Synthesis, CV: Low Level & Physics-based Vision


As a neuromorphic camera with high temporal resolution, spike camera can capture dynamic scenes with high-speed motion. Recently, spike camera with a color filter array (CFA) has been developed for color imaging. There are some methods for spike camera demosaicing to reconstruct color images from Bayer-pattern spike streams. However, the demosaicing results are bothered by severe noise in spike streams, to which previous works pay less attention. In this paper, we propose an iterative joint demosaicing and denoising network (SJDD-Net) for spike cameras based on the observation model. Firstly, we design a color spike representation (CSR) to learn latent representation from Bayer-pattern spike streams. In CSR, we propose an offset-sharing deformable convolution module to align temporal features of color channels. Then we develop a spike noise estimator (SNE) to obtain features of the noise distribution. Finally, a color correlation prior (CCP) module is proposed to utilize the color correlation for better details. For training and evaluation, we designed a spike camera simulator to generate Bayer-pattern spike streams with synthesized noise. Besides, we captured some Bayer-pattern spike streams, building the first real-world captured dataset to our knowledge. Experimental results show that our method can restore clean images from Bayer-pattern spike streams. The source codes and dataset are available at



How to Cite

Dong, Y., Xiong, R., Zhao, J., Zhang, J., Fan, X., Zhu, S., & Huang, T. (2024). Joint Demosaicing and Denoising for Spike Camera. Proceedings of the AAAI Conference on Artificial Intelligence, 38(2), 1582-1590.



AAAI Technical Track on Computer Vision I