XL-WSD: An Extra-Large and Cross-Lingual Evaluation Framework for Word Sense Disambiguation

Authors

  • Tommaso Pasini Sapienza University of Rome
  • Alessandro Raganato University of Helsinki
  • Roberto Navigli Sapienza University of Rome

DOI:

https://doi.org/10.1609/aaai.v35i15.17609

Keywords:

Lexical & Frame Semantics, Semantic Parsing

Abstract

Transformer-based architectures brought a breeze of change to Word Sense Disambiguation (WSD), improving models' performances by a large margin. The fast development of new approaches has been further encouraged by a well-framed evaluation suite for English, which has allowed their performances to be kept track of and compared fairly. However, other languages have remained largely unexplored, as testing data are available for a few languages only and the evaluation setting is rather matted. In this paper, we untangle this situation by proposing XL-WSD, a cross-lingual evaluation benchmark for the WSD task featuring sense-annotated development and test sets in 18 languages from six different linguistic families, together with language-specific silver training data. We leverage XL-WSD datasets to conduct an extensive evaluation of neural and knowledge-based approaches, including the most recent multilingual language models. Results show that the zero-shot knowledge transfer across languages is a promising research direction within the WSD field, especially when considering low-resourced languages where large pre-trained multilingual models still perform poorly. We make the evaluation suite and the code for performing the experiments available at https://sapienzanlp.github.io/xl-wsd/.

Downloads

Published

2021-05-18

How to Cite

Pasini, T., Raganato, A., & Navigli, R. (2021). XL-WSD: An Extra-Large and Cross-Lingual Evaluation Framework for Word Sense Disambiguation. Proceedings of the AAAI Conference on Artificial Intelligence, 35(15), 13648-13656. https://doi.org/10.1609/aaai.v35i15.17609

Issue

Section

AAAI Technical Track on Speech and Natural Language Processing II