Modelling Form-Meaning Systematicity with Linguistic and Visual Features

Authors

  • Arie Soeteman University of Amsterdam
  • Dario Gutierrez IBM Research
  • Elia Bruni University of Amsterdam
  • Ekaterina Shutova University of Amsterdam

DOI:

https://doi.org/10.1609/aaai.v34i05.6416

Abstract

Several studies in linguistics and natural language processing (NLP) pointed out systematic correspondences between word form and meaning in language. A prominent example of such systematicity is iconicity, which occurs when the form of a word is motivated by some perceptual (e.g. visual) aspect of its referent. However, the existing data-driven approaches to form-meaning systematicity modelled word meanings relying on information extracted from textual data alone. In this paper, we investigate to what extent our visual experience explains some of the form-meaning systematicity found in language. We construct word meaning representations from linguistic as well as visual data and analyze the structure and significance of form-meaning systematicity found in English using these models. Our findings corroborate the existence of form-meaning systematicity and show that this systematicity is concentrated in localized clusters. Furthermore, applying a multimodal approach allows us to identify new patterns of systematicity that have not been previously identified with the text-based models.

Downloads

Published

2020-04-03

How to Cite

Soeteman, A., Gutierrez, D., Bruni, E., & Shutova, E. (2020). Modelling Form-Meaning Systematicity with Linguistic and Visual Features. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05), 8870-8877. https://doi.org/10.1609/aaai.v34i05.6416

Issue

Section

AAAI Technical Track: Natural Language Processing