Word Embedding Based Correlation Model for Question/Answer Matching

Authors

  • Yikang Shen Beihang University
  • Wenge Rong Beihang University
  • Nan Jiang Beihang University
  • Baolin Peng The Chinese University of Hong Kong
  • Jie Tang Tsinghua University
  • Zhang Xiong Beihang University

DOI:

https://doi.org/10.1609/aaai.v31i1.11002

Keywords:

QA, word embedding

Abstract

The large scale of Q&A archives accumulated in community based question answering (CQA) servivces are important information and knowledge resource on the web. Question and answer matching task has been attached much importance to for its ability to reuse knowledge stored in these systems: it can be useful in enhancing user experience with recurrent questions. In this paper, a Word Embedding based Correlation (WEC) model is proposed by integrating advantages of both the translation model and word embedding. Given a random pair of words, WEC can score their co-occurrence probability in Q&A pairs, while it can also leverage the continuity and smoothness of continuous space word representation to deal with new pairs of words that are rare in the training parallel text. An experimental study on Yahoo! Answers dataset and Baidu Zhidao dataset shows this new method's promising potential.

Downloads

Published

2017-02-12

How to Cite

Shen, Y., Rong, W., Jiang, N., Peng, B., Tang, J., & Xiong, Z. (2017). Word Embedding Based Correlation Model for Question/Answer Matching. Proceedings of the AAAI Conference on Artificial Intelligence, 31(1). https://doi.org/10.1609/aaai.v31i1.11002