Iterative Data Programming for Expanding Text Classification Corpora


  • Neil Mallinar Pryon Research
  • Abhishek Shah IBM Watson
  • Tin Kam Ho IBM Watson
  • Rajendra Ugrani IBM Watson
  • Ayush Gupta IBM Watson



Real-world text classification tasks often require many labeled training examples that are expensive to obtain. Recent advancements in machine teaching, specifically the data programming paradigm, facilitate the creation of training data sets quickly via a general framework for building weak models, also known as labeling functions, and denoising them through ensemble learning techniques. We present a fast, simple data programming method for augmenting text data sets by generating neighborhood-based weak models with minimal supervision. Furthermore, our method employs an iterative procedure to identify sparsely distributed examples from large volumes of unlabeled data. The iterative data programming techniques improve newer weak models as more labeled data is confirmed with human-in-loop. We show empirical results on sentence classification tasks, including those from a task of improving intent recognition in conversational agents.




How to Cite

Mallinar, N., Shah, A., Ho, T. K., Ugrani, R., & Gupta, A. (2020). Iterative Data Programming for Expanding Text Classification Corpora. Proceedings of the AAAI Conference on Artificial Intelligence, 34(08), 13332-13337.



IAAI Technical Track: Emerging Papers