TY - JOUR AU - Jameel, Shoaib AU - Fu, Zihao AU - Shi, Bei AU - Lam, Wai AU - Schockaert, Steven PY - 2019/07/17 Y2 - 2024/03/28 TI - Word Embedding as Maximum A Posteriori Estimation JF - Proceedings of the AAAI Conference on Artificial Intelligence JA - AAAI VL - 33 IS - 01 SE - AAAI Technical Track: Natural Language Processing DO - 10.1609/aaai.v33i01.33016562 UR - https://ojs.aaai.org/index.php/AAAI/article/view/4624 SP - 6562-6569 AB - <p>The GloVe word embedding model relies on solving a global optimization problem, which can be reformulated as a maximum likelihood estimation problem. In this paper, we propose to generalize this approach to word embedding by considering parametrized variants of the GloVe model and incorporating priors on these parameters. To demonstrate the usefulness of this approach, we consider a word embedding model in which each context word is associated with a corresponding variance, intuitively encoding how informative it is. Using our framework, we can then learn these variances together with the resulting word vectors in a unified way. We experimentally show that the resulting word embedding models outperform GloVe, as well as many popular alternatives.</p> ER -