FocalDreamer: Text-Driven 3D Editing via Focal-Fusion Assembly

Authors

  • Yuhan Li Shanghai Jiao Tong University
  • Yishun Dou Huawei
  • Yue Shi Shanghai Jiao Tong University
  • Yu Lei Shanghai Jiao Tong University
  • Xuanhong Chen Shanghai Jiao Tong University
  • Yi Zhang Shanghai Jiao Tong University
  • Peng Zhou Shanghai Jiao Tong University
  • Bingbing Ni Shanghai Jiao Tong University

DOI:

https://doi.org/10.1609/aaai.v38i4.28113

Keywords:

CV: 3D Computer Vision, CV: Multi-modal Vision

Abstract

While text-3D editing has made significant strides in leveraging score distillation sampling, emerging approaches still fall short in delivering separable, precise and consistent outcomes that are vital to content creation. In response, we introduce FocalDreamer, a framework that merges base shape with editable parts according to text prompts for fine-grained editing within desired regions. Specifically, equipped with geometry union and dual-path rendering, FocalDreamer assembles independent 3D parts into a complete object, tailored for convenient instance reuse and part-wise control. We propose geometric focal loss and style consistency regularization, which encourage focal fusion and congruent overall appearance. Furthermore, FocalDreamer generates high-fidelity geometry and PBR textures which are compatible with widely-used graphics engines. Extensive experiments have highlighted the superior editing capabilities of FocalDreamer in both quantitative and qualitative evaluations.

Published

2024-03-24

How to Cite

Li, Y., Dou, Y., Shi, Y., Lei, Y., Chen, X., Zhang, Y., Zhou, P. ., & Ni, B. (2024). FocalDreamer: Text-Driven 3D Editing via Focal-Fusion Assembly. Proceedings of the AAAI Conference on Artificial Intelligence, 38(4), 3279-3287. https://doi.org/10.1609/aaai.v38i4.28113

Issue

Section

AAAI Technical Track on Computer Vision III