TY - JOUR AU - Qadir, Ashequl AU - Mendes, Pablo AU - Gruhl, Daniel AU - Lewis, Neal PY - 2015/02/19 Y2 - 2024/03/28 TI - Semantic Lexicon Induction from Twitter with Pattern Relatedness and Flexible Term Length JF - Proceedings of the AAAI Conference on Artificial Intelligence JA - AAAI VL - 29 IS - 1 SE - Main Track: NLP and Text Mining DO - 10.1609/aaai.v29i1.9519 UR - https://ojs.aaai.org/index.php/AAAI/article/view/9519 SP - AB - <p> With the rise of social media, learning from informal text has become increasingly important. We present a novel semantic lexicon induction approach that is able to learn new vocabulary from social media. Our method is robust to the idiosyncrasies of informal and open-domain text corpora. Unlike previous work, it does not impose restrictions on the lexical features of candidate terms — e.g. by restricting entries to nouns or noun phrases —while still being able to accurately learn multiword phrases of variable length. Starting with a few seed terms for a semantic category, our method first explores the context around seed terms in a corpus, and identifies context patterns that are relevant to the category. These patterns are used to extract candidate terms — i.e. multiword segments that are further analyzed to ensure meaningful term boundary segmentation. We show that our approach is able to learn high quality semantic lexicons from informally written social media text of Twitter, and can achieve accuracy as high as 92% in the top 100 learned category members. </p> ER -