Switch-GPT: An Effective Method for Constrained Text Generation under Few-Shot Settings (Student Abstract)
Keywords:Few-shot Learning, GPT, NLP, Pre-trained Language Model, Text Generation
AbstractIn real-world applications of natural language generation, target sentences are often required to satisfy some lexical constraints. However, the success of most neural-based models relies heavily on data, which is infeasible for data-scarce new domains. In this work, we present FewShotAmazon, the first benchmark for the task of Constrained Text Generation under few-shot settings on multiple domains. Further, we propose the Switch-GPT model, in which we utilize the strong language modeling capacity of GPT-2 to generate fluent and well-formulated sentences, while using a light attention module to decide which constraint to attend to at each step. Experiments show that the proposed Switch-GPT model is effective and remarkably outperforms the baselines. Codes will be available at https://github.com/chang-github-00/Switch-GPT.
How to Cite
Ma, C., Zhang, S., Shen, G., & Deng, Z. (2022). Switch-GPT: An Effective Method for Constrained Text Generation under Few-Shot Settings (Student Abstract). Proceedings of the AAAI Conference on Artificial Intelligence, 36(11), 13011-13012. https://doi.org/10.1609/aaai.v36i11.21642
AAAI Student Abstract and Poster Program