TY - JOUR AU - Xu, Shusheng AU - Zhang, Xingxing AU - Wu, Yi AU - Wei, Furu PY - 2022/06/28 Y2 - 2024/03/28 TI - Sequence Level Contrastive Learning for Text Summarization JF - Proceedings of the AAAI Conference on Artificial Intelligence JA - AAAI VL - 36 IS - 10 SE - AAAI Technical Track on Speech and Natural Language Processing DO - 10.1609/aaai.v36i10.21409 UR - https://ojs.aaai.org/index.php/AAAI/article/view/21409 SP - 11556-11565 AB - Contrastive learning models have achieved great success in unsupervised visual representation learning, which maximize the similarities between feature representations of different views of the same image, while minimize the similarities between feature representations of views of different images. In text summarization, the output summary is a shorter form of the input document and they have similar meanings. In this paper, we propose a contrastive learning model for supervised abstractive text summarization, where we view a document, its gold summary and its model generated summaries as different views of the same mean representation and maximize the similarities between them during training. We improve over a strong sequence-to-sequence text generation model (i.e., BART) on three different summarization datasets. Human evaluation also shows that our model achieves better faithfulness ratings compared to its counterpart without contrastive objectives. We release our code at https://github.com/xssstory/SeqCo. ER -