Can You Answer This? – Exploring Zero-Shot QA Generalization Capabilities in Large Language Models (Student Abstract)
Keywords:Natural Language Processing, Zero-Shot Learning, Extractive Question Answering
AbstractThe buzz around Transformer-based language models (TLM) such as BERT, RoBERTa, etc. is well-founded owing to their impressive results on an array of tasks. However, when applied to areas needing specialized knowledge (closed-domain), such as medical, finance, etc. their performance takes drastic hits, sometimes more than their older recurrent/convolutional counterparts. In this paper, we explore zero-shot capabilities of large LMs for extractive QA. Our objective is to examine performance change in the face of domain drift i.e. when the target domain data is vastly different in semantic and statistical properties from the source domain and attempt to explain the subsequent behavior. To this end, we present two studies in this paper while planning further experiments later down the road. Our findings indicate flaws in the current generation of TLM limiting their performance on closed-domain tasks.
How to Cite
Sengupta, S., Ghosh, S., Nakov, P., & Mitra, P. (2023). Can You Answer This? – Exploring Zero-Shot QA Generalization Capabilities in Large Language Models (Student Abstract). Proceedings of the AAAI Conference on Artificial Intelligence, 37(13), 16318-16319. https://doi.org/10.1609/aaai.v37i13.27019
AAAI Student Abstract and Poster Program