Accurate Temporal Action Proposal Generation with Relation-Aware Pyramid Network


  • Jialin Gao Shanghai Jiao Tong University
  • Zhixiang Shi Cloudwalk Technology
  • Guanshuo Wang Shanghai Jiao Tong University
  • Jiani Li Cloudwalk Technology
  • Yufeng Yuan Cloudwalk Technology
  • Shiming Ge Institute of Information Engineering, Chinese Academy of Sciences
  • Xi Zhou Cloudwalk Technology



Accurate temporal action proposals play an important role in detecting actions from untrimmed videos. The existing approaches have difficulties in capturing global contextual information and simultaneously localizing actions with different durations. To this end, we propose a Relation-aware pyramid Network (RapNet) to generate highly accurate temporal action proposals. In RapNet, a novel relation-aware module is introduced to exploit bi-directional long-range relations between local features for context distilling. This embedded module enhances the RapNet in terms of its multi-granularity temporal proposal generation ability, given predefined anchor boxes. We further introduce a two-stage adjustment scheme to refine the proposal boundaries and measure their confidence in containing an action with snippet-level actionness. Extensive experiments on the challenging ActivityNet and THUMOS14 benchmarks demonstrate our RapNet generates superior accurate proposals over the existing state-of-the-art methods.




How to Cite

Gao, J., Shi, Z., Wang, G., Li, J., Yuan, Y., Ge, S., & Zhou, X. (2020). Accurate Temporal Action Proposal Generation with Relation-Aware Pyramid Network. Proceedings of the AAAI Conference on Artificial Intelligence, 34(07), 10810-10817.



AAAI Technical Track: Vision