Learning to Detect 3D Facial Landmarks via Heatmap Regression with Graph Convolutional Network

Authors

  • Yuan Wang Institute of Automation, Chinese Academy of Sciences University of Chinese Academy of Sciences
  • Min Cao School of Computer Science and Technology, Soochow University, 215006 Suzhou, China.
  • Zhenfeng Fan Institute of Computing Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences
  • Silong Peng Institute of Automation, Chinese Academy of Sciences University of Chinese Academy of Sciences

DOI:

https://doi.org/10.1609/aaai.v36i3.20161

Keywords:

Computer Vision (CV)

Abstract

3D facial landmark detection is extensively used in many research fields such as face registration, facial shape analysis, and face recognition. Most existing methods involve traditional features and 3D face models for the detection of landmarks, and their performances are limited by the hand-crafted intermediate process. In this paper, we propose a novel 3D facial landmark detection method, which directly locates the coordinates of landmarks from 3D point cloud with a well-customized graph convolutional network. The graph convolutional network learns geometric features adaptively for 3D facial landmark detection with the assistance of constructed 3D heatmaps, which are Gaussian functions of distances to each landmark on a 3D face. On this basis, we further develop a local surface unfolding and registration module to predict 3D landmarks from the heatmaps. The proposed method forms the first baseline of deep point cloud learning method for 3D facial landmark detection. We demonstrate experimentally that the proposed method exceeds the existing approaches by a clear margin on BU-3DFE and FRGC datasets for landmark localization accuracy and stability, and also achieves high-precision results on a recent large-scale dataset.

Downloads

Published

2022-06-28

How to Cite

Wang, Y., Cao, M., Fan, Z., & Peng, S. (2022). Learning to Detect 3D Facial Landmarks via Heatmap Regression with Graph Convolutional Network. Proceedings of the AAAI Conference on Artificial Intelligence, 36(3), 2595-2603. https://doi.org/10.1609/aaai.v36i3.20161

Issue

Section

AAAI Technical Track on Computer Vision III