Representing the Unification of Text Featurization using a Context-Free Grammar

Authors

  • Doruk Kilitcioglu Fidelity Investments
  • Serdar Kadioglu Fidelity Investments

Keywords:

Text Featurization, Embeddings, Natural Language Processing

Abstract

We propose a novel context-free grammar to represent text embeddings in conjunction with their various transformations. We show how this grammar can serve as a unification layer on top of different featurization techniques, and their hybridization thereof. The approach is embodied in an open-source library, called TextWiser, with a high-level user interface to serve researchers and practitioners. The goal of TextWiser is to enable rapid experimentation with various featurization methods and to serve as a building block within AI applications consuming unstructured data. We highlight several key benefits that are desirable especially in industrial settings where rapid experimentation, reusability, reproducibility, and time to market are of great interest. Finally, we showcase a deployed service powered by TextWiser as a proof-of-concept enterprise application.

Downloads

Published

2021-05-18

How to Cite

Kilitcioglu, D., & Kadioglu, S. (2021). Representing the Unification of Text Featurization using a Context-Free Grammar. Proceedings of the AAAI Conference on Artificial Intelligence, 35(17), 15439-15445. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17814

Issue

Section

IAAI Technical Track on Innovative Tools for Enabling AI Application