Huffman Coding for Storing Non-Uniformly Distributed Messages in Networks of Neural Cliques

Authors

  • Bartosz Boguslawski French Alternative Energies and Atomic Energy Commission
  • Vincent Gripon TELECOM Bretagne
  • Fabrice Seguin TELECOM Bretagne
  • Frédéric Heitzmann French Alternative Energies and Atomic Energy Commission

DOI:

https://doi.org/10.1609/aaai.v28i1.8736

Keywords:

neural clique, sparsity, associative memory, non-uniform distribution, compression code

Abstract

Associative memories are data structures that allow retrieval of previously stored messages given part of their content. They thus behave similarly to human brain's memory that is capable for instance of retrieving the end of a song given its beginning. Among different families of associative memories, sparse ones are known to provide the best efficiency (ratio of the number of bits stored to that of bits used). Nevertheless, it is well known that non-uniformity of the stored messages can lead to dramatic decrease in performance. Recently, a new family of sparse associative memories achieving almost-optimal efficiency has been proposed. Their structure induces a direct mapping between input messages and stored patterns. In this work, we show the impact of non-uniformity on the performance of this recent model and we exploit the structure of the model to introduce several strategies to allow for efficient storage of non-uniform messages. We show that a technique based on Huffman coding is the most efficient.

Downloads

Published

2014-06-19

How to Cite

Boguslawski, B., Gripon, V., Seguin, F., & Heitzmann, F. (2014). Huffman Coding for Storing Non-Uniformly Distributed Messages in Networks of Neural Cliques. Proceedings of the AAAI Conference on Artificial Intelligence, 28(1). https://doi.org/10.1609/aaai.v28i1.8736