Structure Learning for Headline Generation


  • Ruqing Zhang Chinese Academy of Sciences (CAS)
  • Jiafeng Guo Chinese Academy of Sciences (CAS)
  • Yixing Fan Chinese Academy of Sciences (CAS)
  • Yanyan Lan Chinese Academy of Sciences (CAS)
  • Xueqi Cheng Chinese Academy of Sciences (CAS)



Headline generation is an important problem in natural language processing, which aims to describe a document by a compact and informative headline. Some recent successes on this task have been achieved by advanced graph-based neural models, which marry the representational power of deep neural networks with the structural modeling ability of the relational sentence graphs. The advantages of graph-based neural models over traditional Seq2Seq models lie in that they can encode long-distance relationship between sentences beyond the surface linear structure. However, since documents are typically weakly-structured data, modern graph-based neural models usually rely on manually designed rules or some heuristics to construct the sentence graph a prior. This may largely limit the power and increase the cost of the graph-based methods. In this paper, therefore, we propose to incorporate structure learning into the graph-based neural models for headline generation. That is, we want to automatically learn the sentence graph using a data-driven way, so that we can unveil the document structure flexibly without prior heuristics or rules. To achieve this goal, we employ a deep & wide network to encode rich relational information between sentences for the sentence graph learning. For the deep component, we leverage neural matching models, either representation-focused or interaction-focused model, to learn semantic similarity between sentences. For the wide component, we encode a variety of discourse relations between sentences. A Graph Convolutional Network (GCN) is then applied over the sentence graph to generate high-level relational representations for headline generation. The whole model could be optimized end-to-end so that the structure and representation could be learned jointly. Empirical studies show that our model can significantly outperform the state-of-the-art headline generation models.




How to Cite

Zhang, R., Guo, J., Fan, Y., Lan, Y., & Cheng, X. (2020). Structure Learning for Headline Generation. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05), 9555-9562.



AAAI Technical Track: Natural Language Processing