Perception Score: A Learned Metric for Open-ended Text Generation Evaluation
Keywords:Generation, General, Applications
AbstractAutomatic evaluation for open-ended natural language generation tasks remains a challenge. We propose a learned evaluation metric: Perception Score. It utilizes a pre-trained model and considers context information for conditional generation. Perception Score assigns a holistic score along with the uncertainty measurement. We conduct experiments on three open-ended conditional generation tasks and two open-ended unconditional generation tasks. Perception Score achieves state-of-the-art results on all the tasks consistently in terms of correlation with human evaluation scores.
How to Cite
Gu, J., Wu, Q., & Yu, Z. (2021). Perception Score: A Learned Metric for Open-ended Text Generation Evaluation. Proceedings of the AAAI Conference on Artificial Intelligence, 35(14), 12902-12910. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/17526
AAAI Technical Track on Speech and Natural Language Processing I