Dialogue for Prompting: A Policy-Gradient-Based Discrete Prompt Generation for Few-Shot Learning


  • Chengzhengxu Li Xi'an Jiaotong University
  • Xiaoming Liu Xi'an Jiaotong University
  • Yichen Wang Xi'an Jiaotong University
  • Duyi Li Xi'an Jiaotong University
  • Yu Lan Xi'an Jiaotong University
  • Chao Shen Xi'an Jiaotong University




NLP: Text Classification, ML: Reinforcement Learning, NLP: (Large) Language Models, NLP: Applications


Prompt-based pre-trained language models (PLMs) paradigm has succeeded substantially in few-shot natural language processing (NLP) tasks. However, prior discrete prompt optimization methods require expert knowledge to design the base prompt set and identify high-quality prompts, which is costly, inefficient, and subjective. Meanwhile, existing continuous prompt optimization methods improve the performance by learning the ideal prompts through the gradient information of PLMs, whose high computational cost, and low readability and generalizability are often concerning. To address the research gap, we propose a Dialogue-comprised Policy-gradient-based Discrete Prompt Optimization (DP_2O) method. We first design a multi-round dialogue alignment strategy for readability prompt set generation based on GPT-4. Furthermore, we propose an efficient prompt screening metric to identify high-quality prompts with linear complexity. Finally, we construct a reinforcement learning (RL) framework based on policy gradients to match the prompts to inputs optimally. By training a policy network with only 0.62M parameters on the tasks in the few-shot setting, DP_2O outperforms the state-of-the-art (SOTA) method by 1.52% in accuracy on average on four open-source datasets. Moreover, subsequent experiments also demonstrate that DP_2O has good universality, robustness and generalization ability.



How to Cite

Li, C., Liu, X., Wang, Y., Li, D., Lan, Y., & Shen, C. (2024). Dialogue for Prompting: A Policy-Gradient-Based Discrete Prompt Generation for Few-Shot Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 38(16), 18481-18489. https://doi.org/10.1609/aaai.v38i16.29809



AAAI Technical Track on Natural Language Processing I