Detailed Facial Geometry Recovery from Multi-View Images by Learning an Implicit Function

Authors

  • Yunze Xiao Nanjing University
  • Hao Zhu Nanjing University
  • Haotian Yang Nanjing University
  • Zhengyu Diao Nanjing University
  • Xiangju Lu iQIYI,Inc.
  • Xun Cao Nanjing University

DOI:

https://doi.org/10.1609/aaai.v36i3.20188

Keywords:

Computer Vision (CV)

Abstract

Recovering detailed facial geometry from a set of calibrated multi-view images is valuable for its wide range of applications. Traditional multi-view stereo (MVS) methods adopt an optimization-based scheme to regularize the matching cost. Recently, learning-based methods integrate all these into an end-to-end neural network and show superiority of efficiency. In this paper, we propose a novel architecture to recover extremely detailed 3D faces within dozens of seconds. Unlike previous learning-based methods that regularize the cost volume via 3D CNN, we propose to learn an implicit function for regressing the matching cost. By fitting a 3D morphable model from multi-view images, the features of multiple images are extracted and aggregated in the mesh-attached UV space, which makes the implicit function more effective in recovering detailed facial shape. Our method outperforms SOTA learning-based MVS in accuracy by a large margin on the FaceScape dataset. The code and data are released in https://github.com/zhuhao-nju/mvfr.

Downloads

Published

2022-06-28

How to Cite

Xiao, Y., Zhu, H., Yang, H., Diao, Z., Lu, X., & Cao, X. (2022). Detailed Facial Geometry Recovery from Multi-View Images by Learning an Implicit Function. Proceedings of the AAAI Conference on Artificial Intelligence, 36(3), 2839-2847. https://doi.org/10.1609/aaai.v36i3.20188

Issue

Section

AAAI Technical Track on Computer Vision III