Action Recognition and State Change Prediction in a Recipe Understanding Task Using a Lightweight Neural Network Model (Student Abstract)

Authors

  • Qing Wan Texas A&M University
  • Yoonsuck Choe Texas A&M University

DOI:

https://doi.org/10.1609/aaai.v34i10.7245

Abstract

Consider a natural language sentence describing a specific step in a food recipe. In such instructions, recognizing actions (such as press, bake, etc.) and the resulting changes in the state of the ingredients (shape molded, custard cooked, temperature hot, etc.) is a challenging task. One way to cope with this challenge is to explicitly model a simulator module that applies actions to entities and predicts the resulting outcome (Bosselut et al. 2018). However, such a model can be unnecessarily complex. In this paper, we propose a simplified neural network model that separates action recognition and state change prediction, while coupling the two through a novel loss function. This allows learning to indirectly influence each other. Our model, although simpler, achieves higher state change prediction performance (67% average accuracy for ours vs. 55% in (Bosselut et al. 2018)) and takes fewer samples to train (10K ours vs. 65K+ by (Bosselut et al. 2018)).

Downloads

Published

2020-04-03

How to Cite

Wan, Q., & Choe, Y. (2020). Action Recognition and State Change Prediction in a Recipe Understanding Task Using a Lightweight Neural Network Model (Student Abstract). Proceedings of the AAAI Conference on Artificial Intelligence, 34(10), 13945-13946. https://doi.org/10.1609/aaai.v34i10.7245

Issue

Section

Student Abstract Track