Brain Decoding Using fNIRS

Authors

  • Lu Cao Singapore University of Technology and Design
  • Dandan Huang School of Engineering, Westlake University, China Institute of Advanced Technology, Westlake Institute for Advanced Study, China
  • Yue Zhang School of Engineering, Westlake University, China Institute of Advanced Technology, Westlake Institute for Advanced Study, China
  • Xiaowei Jiang Henan University
  • Yanan Chen Henan University

Keywords:

Psycholinguistics and Language Learning

Abstract

Brain activation can reflect semantic information elicited by natural words and concepts. Increasing research has been conducted on decoding such neural activation patterns using representational semantic models. However, prior work decoding semantic meaning from neurophysiological responses has been largely limited to ECoG, fMRI, MEG, and EEG techniques, each having its own advantages and limitations. More recently, the functional near infrared spectroscopy (fNIRS) has emerged as an alternative hemodynamic-based approach and possesses a number of strengths. We investigate brain decoding tasks under the help of fNIRS and empirically compare fNIRS with fMRI. Primarily, we find that: 1) like fMRI scans, activation patterns recorded from fNIRS encode rich information for discriminating concepts, but show limits on the possibility of decoding fine-grained semantic clues; 2) fNIRS decoding shows robustness across different brain regions, semantic categories and even subjects; 3) fNIRS has higher accuracy being decoded based on multi-channel patterns as compared to single-channel ones, which is in line with our intuition of the working mechanism of human brain. Our findings prove that fNIRS has the potential to promote a deep integration of NLP and cognitive neuroscience from the perspective of language understanding. We release the largest fNIRS dataset by far to facilitate future research.

Downloads

Published

2021-05-18

How to Cite

Cao, L., Huang, D., Zhang, Y., Jiang, X., & Chen, Y. (2021). Brain Decoding Using fNIRS. Proceedings of the AAAI Conference on Artificial Intelligence, 35(14), 12602-12611. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17493

Issue

Section

AAAI Technical Track on Speech and Natural Language Processing I