Journey to the Center of the Knowledge Neurons: Discoveries of Language-Independent Knowledge Neurons and Degenerate Knowledge Neurons

Authors

  • Yuheng Chen Institute of Automation, Chinese Academy of Sciences University of Chinese Academy of Sciences
  • Pengfei Cao Institute of Automation, Chinese Academy of Sciences University of Chinese Academy of Sciences
  • Yubo Chen Institute of Automation, Chinese Academy of Sciences University of Chinese Academy of Sciences
  • Kang Liu Institute of Automation, Chinese Academy of Sciences University of Chinese Academy of Sciences
  • Jun Zhao Institute of Automation, Chinese Academy of Sciences University of Chinese Academy of Sciences

DOI:

https://doi.org/10.1609/aaai.v38i16.29735

Keywords:

NLP: Interpretability, Analysis, and Evaluation of NLP Models, NLP: (Large) Language Models

Abstract

Pre-trained language models (PLMs) contain vast amounts of factual knowledge, but how the knowledge is stored in the parameters remains unclear. This paper delves into the complex task of understanding how factual knowledge is stored in multilingual PLMs, and introduces the Architecture-adapted Multilingual Integrated Gradients method, which successfully localizes knowledge neurons more precisely compared to current methods, and is more universal across various architectures and languages. Moreover, we conduct an in-depth exploration on knowledge neurons, leading to the following two important discoveries: (1) The discovery of Language-Independent Knowledge Neurons, which store factual knowledge in a form that transcends language. We design cross-lingual knowledge editing experiments, demonstrating that the PLMs can accomplish this task based on language-independent neurons; (2) The discovery of Degenerate Knowledge Neurons, a novel type of neuron showing that different knowledge neurons can store the same fact. Its property of functional overlap endows the PLMs with a robust mastery of factual knowledge. We design fact-checking experiments, proving that the degenerate knowledge neurons can help the PLMs to detect wrong facts. Experiments corroborate these findings, shedding light on the mechanisms of factual knowledge storage in multilingual PLMs, and contribute valuable insights to the field. The code is available at https://github.com/heng840/AMIG.

Published

2024-03-24

How to Cite

Chen, Y., Cao, P., Chen, Y., Liu, K., & Zhao, J. (2024). Journey to the Center of the Knowledge Neurons: Discoveries of Language-Independent Knowledge Neurons and Degenerate Knowledge Neurons. Proceedings of the AAAI Conference on Artificial Intelligence, 38(16), 17817-17825. https://doi.org/10.1609/aaai.v38i16.29735

Issue

Section

AAAI Technical Track on Natural Language Processing I