TY - JOUR AU - Bisk, Yonatan AU - Zellers, Rowan AU - Le bras, Ronan AU - Gao, Jianfeng AU - Choi, Yejin PY - 2020/04/03 Y2 - 2024/03/29 TI - PIQA: Reasoning about Physical Commonsense in Natural Language JF - Proceedings of the AAAI Conference on Artificial Intelligence JA - AAAI VL - 34 IS - 05 SE - AAAI Technical Track: Natural Language Processing DO - 10.1609/aaai.v34i05.6239 UR - https://ojs.aaai.org/index.php/AAAI/article/view/6239 SP - 7432-7439 AB - <p>To apply eyeshadow without a brush, should I use a <em>cotton swab or a toothpick</em>? Questions requiring this kind of <strong>physical commonsense</strong> pose a challenge to today's natural language understanding systems. While recent pretrained models (such as BERT) have made progress on question answering over more <em>abstract</em> domains – such as news articles and encyclopedia entries, where text is plentiful – in more <em>physical</em> domains, text is inherently limited due to reporting bias. Can AI systems learn to reliably answer physical commonsense questions without experiencing the physical world?</p><p>In this paper, we introduce the task of physical commonsense reasoning and a corresponding benchmark dataset <strong>Physical Interaction: Question Answering</strong> or <strong>PIQA</strong>. Though humans find the dataset easy (95% accuracy), large pretrained models struggle (∼75%). We provide analysis about the dimensions of knowledge that existing models lack, which offers significant opportunities for future research.</p> ER -